Frustrating experience with AWS Beanstalk

I have used Azure's Paas services before. For a friend's website, I was trying out AWS's Paas offering today. The experience is nothing less than just frustrating. I have a fully functioning rails application. It is working fine locally. However, when I tried to upload the application using the beanstalk command line, I am seeing the following error.

ERROR: [Instance: i-002808929ac3e534b] Command failed on instance. Return code: 1 Output: (TRUNCATED)...SyntaxError: Invalid CSS after "": expected selector or at-rule, was "{"
(sass):1
/opt/rubies/ruby-2.3.1/bin/bundle:23:in `load'
/opt/rubies/ruby-2.3.1/bin/bundle:23:in `

'
Tasks: TOP => assets:precompile
(See full trace by running task with --trace).
Hook /opt/elasticbeanstalk/hooks/appdeploy/pre/11_asset_compilation.sh failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.

Now the complete log is not available here. You need to log into the instance and check it. However, as the lines show, it is a problem with asset pre-compilation. I tried running asset compilation locally; but I do not see this error. Interestingly, the instance created by the CLI did not allow permission for SSH. It's acceptable source was set to some other IP. It seems to me they launched their product in a hurry.

SVN export using capistrano

If you are using SVN for version control and

capistrano for deployments, capistrano does code checkout from the application machines. It is often preferable to use svn export instead of svn checkout. Capistrano defaults to checkout, but it can be configured to use export by setting the :deploy_via parameter to 'export' in deployment script.

Rails not detecting iconv

Whenever I started rails console or server, I was getting the following line about iconv.

no iconv, you might want to look into it.

I tried installing iconv package followed by the gem. Yet, I was unable to get rid of the message. I was not bothered much about the message till I started using rubyzip gem for creating some archives as it requires iconv gem. The solution is quite simple actually. Just add iconv to the Gemfile and rubyzip works fine. Also, the error line disappears. Of course, this is just a by-pass to get things working. I might go for an analysis some time later. Suggestions for that are welcome.

[Rails:] false.present? being false is not intuitive (to me at least)

In Rails I often use the present? method to check whether a key-value pair is present is an input hash. The code would be something like the following.

def trial_method(params)
    o = Model.where(:id => params[:id].to_i).first
    o.field = params[:field_key] if params[:field_key].present?
    o.save!
end
This works fine when field is most cases. However, when the field is a boolean field, there is a subtle with this code.

When params[:field_key] is true, the code works fine; but whenever it is false, the code fails. The intent is to see if the key-value pair is present and if present the value should be assigned to the field. Now, the value can be both true and false and both of those should be assigned to field. However, when the value is false, the present? function returns false and the field is not set any more. So the field value stays true and with the above code can not be flipped to false. The correction is of course simple.

o.field = params[:field_key] unless params[:field_key].nil?

This behaviour did not seem intuitive to me. So, I looked up documentation and source for present?. The definition of present? is quite simple.

def present?
  !blank?
end
Clearly all it does is to negate the result of blank?. So, I looked up the definition for that too.

def blank?
  respond_to?(:empty?) ? empty? : !self
end

When I tested false.respond_to?(:empty?), it returned false. So, it is clearly executing the !self part, which translates to false.blank? being true and therefore false.present? being false.

Bypassing ActiveRecord cache

ActiveRecord is the default object relational model of the Rails web framework. It obviously follows the active record architectural pattern. Now, ActiveRecord maintains it own query cache which is different from the query cache of the underlying database server. This query cache is a rather simplistic one.

The issue that brought the requirement of query cache bypassing into picture was as follows.

1. a call to first object of the model to check if any records existed (Model.first)
2. raw SQL query to truncate the table
3. a call again to first object of the model to check if any records existed (Model.first)

Now, #1 and #3 obviously generated same SQL query. So, ActiveRecord served #3 from its cache.

We found multiple approaches of bypassing the ActiveRecord cache.

Approach 1
Clear the entire ActiveRecord cache. In Rails 2 this can be done using

ActiveRecord::Base.query_cache.clear_query_cache

In Rails 3, the same can be achieved using the following line.

 ActiveRecord::Base.connection.clear_query_cache

This approach however would clear the entire ActiveRecord cache which in production environment means increasing load on database server which is already the bottleneck. Plus, this approach is like using a jack hammer where a finger-tap would work.

Approach 2
This approach exploits the simplicity of the ActiveRecord cache. It forms the query in such a manner that the query string is very likely to be different from previous queries.

r = call random number generator
where_clause = "r = r"

Appending the above where clause to #1 and #3, we obtained queries that are very likely to be different from previous ones at least within the life time of the cache. This approach is obviously not elegant.

Approach 3
In this approach, we went with raw SQL queries not only for #2 but also for #1 and #3. ActiveRecord does not seem to cache raw SQL queries. So we could replace the call to the 'first' method of the model with something similar to the following.


sql = "select * from table_name limit 1"
ActiveRecord::Base.connection.execute sql

Although we can get the job done by this approach, it is bad practice to execute raw SQL.

Approach 4
Finally, we found a way of doing it through Rails. We need to explicitly tell Rails not to serve our queries from its cache for #1 and #3. This can be done as follows.

Model.uncached do
    Model.first
end

As this method does the job using the Rails framework, the abstraction all provided by ActiveRecord remains unbroken.

Rails is re-inventing the wheel

In recent years, there has been a surge in web applications. To support the growing market, frameworks have developed around scripting languages for developing web applications fast. Of those Ruby on Rails seems to be the most mature. Django, the most advanced Python framework for web applications is yet to come at par with it.

Be it Django or Ruby on Rails, both follow a model-view-controller architecture to web applications. Now, many web applications using these frameworks follow 'skinny controllers, fat models'. As a result the models become home to a lot of business logic. Models are intended to serve only as an abstraction to the database. They are meant to be 'models of data'. Also, when the application grows huge, we run into issues of scalability and we resort to techniques like sharding. Now, there are of course multiple ways of sharding and you need to decide what suits you better. Here, I list a few ways for Rails applications.

• data_fabric
• octopus
• Spock proxy

Now, sometimes the sharding logic also creeps into the models. Obviously objects of model classes do not have a single purpose any more. Over time, the model classes obviously become too complicated. It becomes difficult to maintain and debug. The Rails community is becoming increasingly aware of these issues. There are multiple ways in which developers are trying to move business logic away from models. Here I list of couple of those ways.

• use cases
• concerns

Looking at the overall scenario reminds me of building N-tier applications using Java Enterprise Edition. The sharding logic and details of data fetching can be a tier below models and the business logic can be a tier above it. ' seems to me developers using web frameworks like Django and Rails are just re-inventing the wheel. Also, Java has optimized garbage collection which can be tuned to various needs. In Ruby garbage collection seems to be a big issue, though good work in the area is coming in the next version.

So much in the name of innovation, eh? In case my ideas seem to waver from reality do point me in the right direction.

Update: You can follow the discussion on Hacker News on the topic.

Rails time formats

During programming, I do tend to forget the various time formats available in Rails. Also, there are so many options available that a simple typo like capitalizing (or not) changes meaning. It is best that this part be always tested properly.

UTF-8 encoded Rails Console in linux

Although, initially while working on Rails project, we do not pay much attention to the character encoding, soon it bites us bad. When we try to play with a string containing Unicode characters in an ASCII encoded Rails console, it complains about it and we are stuck.

It is quite easy to get a UTF-8 encoded Rails console. All we need to do is properly set the LANG environment variable of the terminal first.

export LANG=en_US.UTF-8

Rails picks up this variable and sets its encoding. So, when we fire up Rails console now and enter the following command, it shows that we have UTF-8 encoded console.

puts __ENCODING__

Now, we can easily play with UTF-8 strings.

Showing tmp folder in Rubymine

I have been using RubyMine for a few months now. I must say it has served me well. Recently, I was writing and reading files in the /tmp folder of root directory of my Rails project. However, in my project pane, it was not showing the /tmp folder. So, I had to go to the folder using my file manager and check the files. When I was through with my task, I wanted to figure out why that folder was not shown in the project pane. Getting the folder to show up was easy. In the project settings, in "Project Structure" section, there is a list of excluded folders. We just have to remove the entry for the /tmp folder.

Setting environment variables in ruby on rails

I use GMail's SMTP gateway for sending out system generated emails in my RoR project. Once, I got everything working, I thought of deploying it on Heroku. However, I host my code on Github and did not want to commit the id and password on the GMail account. So, I decided to read the email id and password as environment variables.

I set the environment variables on Heroku using the following:

heroku config:add ID=my.id@gmail.com
heroku config:add PASS=mypass

In the code I accessed these variables using ENV['ID'] and ENV['PASS'] respectively. So, the code was somewhat as follows:

class Emailer < ActionMailer::Base
    default from: "#{ENV['ID']}", :charset => "UTF-8"
    # other methods
end

With this I could easily send out mails from Heroku.

Whenever I want to test anything on my local using emails, I set environment variables before running the Rails server.

export ID=my.id@gmail.com
export PASS=mypass
rails s