Correct do...while in Ruby

Rubyists use the .each or .map way of looping frequently. However, when there is a need of a while loop, the following is the correct way.
loop do
    # statements
    break if condition
end

Invoke a rake task multiple times from another

To invoke a rake task within another, one can do the following:

Rake::Task['namespace:task_name'].invoke

However, if you need to run a task multiple times, the following will not work.

n.times do
    Rake::Task['namespace:task_name'].invoke
end

It only executes the task once. This behaviour is useful when you are loading dependent tasks. For example, if there are two tasks that depend upon :environment task, this behaviour ensures :environment is loaded only once. However, the above code is written with a different intension. To make it work correctly, the rake task has to be re-enabled. The way to do it is as follows:

n.times do
    task = Rake::Task['namespace:task_name'].invoke
    task.reenable
    task.invoke
end

Writing files in a specific encoding in Ruby

Recently, I was trying to create PDF files of barcodes. I was using barby gem for generating barcodes. When I tried to write the generated PDF, I found that the PDF string was in ASCII-8BIT encoding while my console was in UTF-8 which was causing Ruby to attempt writing in UTF-8 encoding causing errors. So, to write to the file in ASCII-8BIT, I had to specify that while opening the file. I achieved that using the following and I could write to the file correctly.

f = File.open("test.pdf", "w:ASCII-8BIT")

f.write the_pdf_string

f.close

Private class methods in Ruby

In Ruby instance methods can be marked as private by declaring them below the private section. However, the same does not work for class methods.

class SomeClass

  def method1
    # do something
  end

  private

  def method2
    # do something els
  end

end

The above works fine. Instances (objects) of the class can not call method2 directly.

class SomeOtherClass

  def self.method1
    # do something
  end

  private

  def self.method2
    # do something else
  end

end

In this case however, one can easily call method2 as follows.

SomeOtherClass.method2

To mark class methods as private,  we will need to use private_class_method as shown below.

class SomeOtherClass

  def self.method1
    # do something
  end

  private_class_method :method2 :method3

  def self.method2
    # do something else
  end

  def self.method3
    # do some other things here
  end


end

Disable compression in Net::HTTP

Recently while upgrading to Ruby 2, I found a JSON API call was failing intermittently. The code was failing when I was trying to parse the response using JSON.parse. Investigating further, I found that the call was successful; but for some calls the response was different than the response for others. Looking at the response, I found it was not JSON at all. It looked like the following.

\x8B\x00\x00\x8C...

I had no clue as to what to make out of such a response. Reading up on it, I found out that the above might be a compressed response. It appeared that the server had the liberty of choosing whether to compress the response or not and whenever the response was uncompressed, my code was working fine. Looking into improvements introduced in Ruby 2, I found that Net::HTTP now automatically requests gzip and deflate compression by default. All I needed to do was to stop doing that and ask for uncompressed response. As my response was rather small, it would hardly matter. To request uncompressed response, I just needed to add the following header to my requests.

'Accept-Encoding' => 'identity'

Gem installation fails due to packaged gems

Rake 0.9.6 comes bundled with ruby 2. However, I needed to build ruby-debug-ide 0.4.17.beta16 against rake 0.8.7. I tried uninstalling rake 0.9.6 but there is no way to do it. Using the following command does not list 0.9.6 as an option.

gem uninstall rake

So, now I had to find a way to tell the gem utility to use rake 0.8.7 instead of 0.9.6. A little reading showed that setting the RAKE environment variable could do this.

RAKE=/usr/lib/ruby/gems/2.0.0/gems/rake-0.8.7/bin/rake gem install ruby-debug-ide -v 0.4.17.beta16 --pre

The above  did not work for some reason. After that I tried the following and it worked.

RAKE=`bundle exec rake` gem install ruby-debug-ide -v 0.4.17.beta16 --pre

SVN export using capistrano

If you are using SVN for version control and

capistrano for deployments, capistrano does code checkout from the application machines. It is often preferable to use svn export instead of svn checkout. Capistrano defaults to checkout, but it can be configured to use export by setting the :deploy_via parameter to 'export' in deployment script.

Rails not detecting iconv

Whenever I started rails console or server, I was getting the following line about iconv.

no iconv, you might want to look into it.

I tried installing iconv package followed by the gem. Yet, I was unable to get rid of the message. I was not bothered much about the message till I started using rubyzip gem for creating some archives as it requires iconv gem. The solution is quite simple actually. Just add iconv to the Gemfile and rubyzip works fine. Also, the error line disappears. Of course, this is just a by-pass to get things working. I might go for an analysis some time later. Suggestions for that are welcome.

[Rails:] false.present? being false is not intuitive (to me at least)

In Rails I often use the present? method to check whether a key-value pair is present is an input hash. The code would be something like the following.

def trial_method(params)
    o = Model.where(:id => params[:id].to_i).first
    o.field = params[:field_key] if params[:field_key].present?
    o.save!
end
This works fine when field is most cases. However, when the field is a boolean field, there is a subtle with this code.

When params[:field_key] is true, the code works fine; but whenever it is false, the code fails. The intent is to see if the key-value pair is present and if present the value should be assigned to the field. Now, the value can be both true and false and both of those should be assigned to field. However, when the value is false, the present? function returns false and the field is not set any more. So the field value stays true and with the above code can not be flipped to false. The correction is of course simple.

o.field = params[:field_key] unless params[:field_key].nil?

This behaviour did not seem intuitive to me. So, I looked up documentation and source for present?. The definition of present? is quite simple.

def present?
  !blank?
end
Clearly all it does is to negate the result of blank?. So, I looked up the definition for that too.

def blank?
  respond_to?(:empty?) ? empty? : !self
end

When I tested false.respond_to?(:empty?), it returned false. So, it is clearly executing the !self part, which translates to false.blank? being true and therefore false.present? being false.

Uninstalling all gems

Some time back I was trying out jruby. I had installed multiple gems for it. Today, while cleaning up my system I was trying to get rid of it because it does not have much use to me. However, uninstalling the gems one by one is a pain. So, I wrote the following line to uninstall all gems.

jruby -S gem list | awk -F ' ' '{print $1}' | xargs jruby -S gem uninstall

P.S. : To understand how to construct such one liners please look at my previous post.

Call by reference in Ruby

In Ruby, some objects are passed by value and some by reference. In case, we want to pass by reference some object which by default gets passed by value, we can just use the id value of the object in Ruby's memory, i.e. the object space. The id can be obtained as follows.

some_object = "some intialization"
some_object.object_id

Here is how pass by reference would work using object id values.

def caller
    blah = "blahblueblah"
    callee(blah.object_id)
end

def callee(object_id)
    puts ObjectSpace._id2ref(doc.object_id)
end

Such use should be highly improbable and might even turn out to be bad practice; but sometimes we just want to do things for the heck of it.

Bypassing ActiveRecord cache

ActiveRecord is the default object relational model of the Rails web framework. It obviously follows the active record architectural pattern. Now, ActiveRecord maintains it own query cache which is different from the query cache of the underlying database server. This query cache is a rather simplistic one.

The issue that brought the requirement of query cache bypassing into picture was as follows.

1. a call to first object of the model to check if any records existed (Model.first)
2. raw SQL query to truncate the table
3. a call again to first object of the model to check if any records existed (Model.first)

Now, #1 and #3 obviously generated same SQL query. So, ActiveRecord served #3 from its cache.

We found multiple approaches of bypassing the ActiveRecord cache.

Approach 1
Clear the entire ActiveRecord cache. In Rails 2 this can be done using

ActiveRecord::Base.query_cache.clear_query_cache

In Rails 3, the same can be achieved using the following line.

 ActiveRecord::Base.connection.clear_query_cache

This approach however would clear the entire ActiveRecord cache which in production environment means increasing load on database server which is already the bottleneck. Plus, this approach is like using a jack hammer where a finger-tap would work.

Approach 2
This approach exploits the simplicity of the ActiveRecord cache. It forms the query in such a manner that the query string is very likely to be different from previous queries.

r = call random number generator
where_clause = "r = r"

Appending the above where clause to #1 and #3, we obtained queries that are very likely to be different from previous ones at least within the life time of the cache. This approach is obviously not elegant.

Approach 3
In this approach, we went with raw SQL queries not only for #2 but also for #1 and #3. ActiveRecord does not seem to cache raw SQL queries. So we could replace the call to the 'first' method of the model with something similar to the following.


sql = "select * from table_name limit 1"
ActiveRecord::Base.connection.execute sql

Although we can get the job done by this approach, it is bad practice to execute raw SQL.

Approach 4
Finally, we found a way of doing it through Rails. We need to explicitly tell Rails not to serve our queries from its cache for #1 and #3. This can be done as follows.

Model.uncached do
    Model.first
end

As this method does the job using the Rails framework, the abstraction all provided by ActiveRecord remains unbroken.

Memory snapshot in Jetbrains RubyMine on linux

I use RubyMine on linux for Ruby on Rails development. Of late, it had been hanging up frequently. I reported this to Jetbrains. They got back to me asking me to provide more details so that the concerned developer can try and find more details about the issue. On a side note, their customer support was so fast in responding, I was amazed. Also, the guy responding back was pretty technical himself.

Getting back to the topic, they had asked me to provide a memory snapshot. The process of generating a memory snapshot is described here. However, that process requires users to download YourKit Java Profiler, which apart from being a large download, comes with a 15-day license. To me it did not make sense. So, I got back to them about it and it turns out on linux, you don't really need it. The linux version of RubyMine, comes with the profiler libraries bundled.

$RUBYMINE_HOME/bin/libyjpagent-linux.so
$RUBYMINE_HOME/bin/libyjpagent-linux64.so

To enable its usage all you need to do is, edit the following script and set IS_EAP to "true".

$RUBYMINE_HOME/bin/rubymine.sh

Restarting RubyMine after that change, will show the memory and CPU snapshot icons. It is also advised to provide thread dumps, as described here, along with the memory snapshot.

Rails is re-inventing the wheel

In recent years, there has been a surge in web applications. To support the growing market, frameworks have developed around scripting languages for developing web applications fast. Of those Ruby on Rails seems to be the most mature. Django, the most advanced Python framework for web applications is yet to come at par with it.

Be it Django or Ruby on Rails, both follow a model-view-controller architecture to web applications. Now, many web applications using these frameworks follow 'skinny controllers, fat models'. As a result the models become home to a lot of business logic. Models are intended to serve only as an abstraction to the database. They are meant to be 'models of data'. Also, when the application grows huge, we run into issues of scalability and we resort to techniques like sharding. Now, there are of course multiple ways of sharding and you need to decide what suits you better. Here, I list a few ways for Rails applications.

• data_fabric
• octopus
• Spock proxy

Now, sometimes the sharding logic also creeps into the models. Obviously objects of model classes do not have a single purpose any more. Over time, the model classes obviously become too complicated. It becomes difficult to maintain and debug. The Rails community is becoming increasingly aware of these issues. There are multiple ways in which developers are trying to move business logic away from models. Here I list of couple of those ways.

• use cases
• concerns

Looking at the overall scenario reminds me of building N-tier applications using Java Enterprise Edition. The sharding logic and details of data fetching can be a tier below models and the business logic can be a tier above it. ' seems to me developers using web frameworks like Django and Rails are just re-inventing the wheel. Also, Java has optimized garbage collection which can be tuned to various needs. In Ruby garbage collection seems to be a big issue, though good work in the area is coming in the next version.

So much in the name of innovation, eh? In case my ideas seem to waver from reality do point me in the right direction.

Update: You can follow the discussion on Hacker News on the topic.

Rails time formats

During programming, I do tend to forget the various time formats available in Rails. Also, there are so many options available that a simple typo like capitalizing (or not) changes meaning. It is best that this part be always tested properly.

Showing tmp folder in Rubymine

I have been using RubyMine for a few months now. I must say it has served me well. Recently, I was writing and reading files in the /tmp folder of root directory of my Rails project. However, in my project pane, it was not showing the /tmp folder. So, I had to go to the folder using my file manager and check the files. When I was through with my task, I wanted to figure out why that folder was not shown in the project pane. Getting the folder to show up was easy. In the project settings, in "Project Structure" section, there is a list of excluded folders. We just have to remove the entry for the /tmp folder.

Setting environment variables in ruby on rails

I use GMail's SMTP gateway for sending out system generated emails in my RoR project. Once, I got everything working, I thought of deploying it on Heroku. However, I host my code on Github and did not want to commit the id and password on the GMail account. So, I decided to read the email id and password as environment variables.

I set the environment variables on Heroku using the following:

heroku config:add ID=my.id@gmail.com
heroku config:add PASS=mypass

In the code I accessed these variables using ENV['ID'] and ENV['PASS'] respectively. So, the code was somewhat as follows:

class Emailer < ActionMailer::Base
    default from: "#{ENV['ID']}", :charset => "UTF-8"
    # other methods
end

With this I could easily send out mails from Heroku.

Whenever I want to test anything on my local using emails, I set environment variables before running the Rails server.

export ID=my.id@gmail.com
export PASS=mypass
rails s

Putting data from a tab separated(.tsv) file into MySQL database

Many a times we need to populate data dumped by someone into a tab-separated file into our database. It can be done using the following ruby code:

   begin
      row = []
      File.open("/path/to/file.tsv") do |f|
        f.each_line do |tsv|
          tsv.chomp!
          row << tsv.split(/\t/)
          method_to_store_detail(row)
        end
      end
    rescue Exception => e
       puts "------exception------#{e.inspect}"
    end


However, the file runs into hundreds of MBs, this will be way too slow. Instead we can use the following MySQL query.


LOAD DATA LOCAL INFILE '/path/to/file' REPLACE INTO TABLE table_name IGNORE 1 LINES (column1, column2, column3, column4);

The "IGNORE 1 LINES" part ensures that the first line containing the header is ignored. In case there are no headers, this part may be excluded. Also, if relative file paths are to be used, the LOCAL keyword may be dropped. This process is way faster than any other process; but validations are bypassed.

Perl REPL

While trying to translate a Perl script to a ruby script, I found the necessity of testing things on the interpreter. Ruby provides IRB for that. If you are using Ruby on Rails, you might prefer Rails console over IRB. However, I could not find anything similar in Perl. I could do some tasks from the command line as follows:

perl -e "print 'Hello World';"

However, whenever I was trying any assignment, it was not working. For example,

perl -e "$k = 'Hello World';print $k;"

did not work. I later came to know it was because bash was gulping $k because I had used a double-quoted string. Using a single-quoted string works fine.

perl -e '$k = "Hello"; print $k;'

A quick search showed that I can use Devel::REPL for this purpose. So, I obtained it from CPAN.


cpan -i Devel::REPL

I prefer using the Perl REPL interpreter using the re.pl script.

[blog@domain ~]$ re.pl

$ my $k = "Hello World"

Hello World$ print $k;

1$ Hello World

$ 

Customizing system-wide gem install

I hardly use gem documentation and mostly install with --no-ri and --no-rdoc options. So instead of typing these options each time I install a new gem, I decided to apply these by default. Adding the following to /etc/gemrc achieved it.

gem: --no-ri --no-rdoc