Monday, 14 September 2020

Gitk permissions on Alacritty

Recently, I started using Alacritty instead of the Terminal app provided by Apple. It sure is much faster.

There seems to be a new and annoying security feature introduced by Apple that has granular permissions for applications downloaded from the internet. So, Alacritty does not start with the same permissions as Terminal. Setting that up is a bit of a hassle. When I started gitk, it asked for some permission and I could not understand it then so I denied it. So, gitk did not work. I thought restarting Alacritty might request permissions again but that is not the case. The error I was getting is as follows:

 % gitk
Error in startup script: 58:102: execution error: Not authorised to send Apple events to System Events. (-1743)
    while executing
"exec osascript -e [format {
        tell application "System Events"
            set frontmost of processes whose unix id is %d to true
        end te..."
    invoked from within
"if {[tk windowingsystem] eq "aqua"} {
    exec osascript -e [format {
        tell application "System Events"
            set frontmost of processes ..."
    (file "/usr/local/bin/gitk" line 12261)

So, I had to find the right permission and grant that to get gitk working. The System Events permission is apparently under Automation section. Once that is granted, Alacritty can run gitk.

Monday, 20 January 2020

Learning Puppet III: Installing modules

In my previous post, we took a look at creating classes in Puppet and applying them locally. Let us look at installing Puppet modules next. Puppet does not install stdlib by default unlike other programming environments.

Common functionalities like reading a JSON file are part of stdlib module. It can often come in handy. Installing it is straightforward. However, after installation the modules are not loaded at run time.

Let us consider an example of reading a JSON file in a Puppet manifest. It can be an example of reading some config from an external resource.

Let us consider the module mod which we created in the previous post.

class mod::test_json{
      include mod

      $hash = loadjson('/tmp/load.json')
      [$message, $return_value] = $hash['output']
      notice("$message, $return_value")

}


To test, let us create a file at with the following content.

{
  "output": {
    "message": "The value is: ",
    "return_value": "0"
  }
}


To apply the manifest, we can try running the following:

puppet apply --modulepath=/path/to/parent/of/module -e "include mod::test_json"

This fails because loadjson is in stdlib. We need to enhance the module path to include stdlib as well. Modules are installed to the path /etc/puppetlabs/code/environments/production/modules. So, we need to modify our command as follows:

puppet apply --modulepath=/etc/puppetlabs/code/environments/production/modules:/vagrant/puppet -e "include elk::test_json"

Learning Puppet - II: Applying Classes

In my previous post about learning Puppet, I left a few open questions that I will attempt to answer as I continue learning.
  1. How can I apply Puppet classes?
  2. How can I install modules?
  3. How can I define functions in Puppet?
In the last post, I was unable to apply a class so I moved ahead without it. Let us look at that in this post.

To apply a class, I created a module, created its init.pp file and added a class in it.

Let us consider the module name to be mod. The init.pp will be located at mod/manifests/init.pp. I just keep it empty to begin with. The name of the class in that file has to match the module name though.

class mod{
}


Moving on to create a class of interest, let us add one to install Openjdk. The file would be mod/manifests/jdk.pp.

class mod::jdk () {
  notice("Installing openJDK")
  # Install java
  package { 'openjdk':
    ensure => installed,
    name => 'java-1.8.0-openjdk-headless',
    alias => 'openjdk',
  }
}


To apply the manifest, the command would be as follows:

puppet apply --modulepath=/path/to/parent/of/module -e "include mod::jdk"

Wednesday, 18 December 2019

jruby-launcher upgrade

Logstash uses JRuby. So, while trying to read the code regarding an issue I was having in Logstash, I had installed JRuby and I had set up multiple Ruby versions using rbenv.

To check whether the right version of Ruby is being used, I tried ruby -v. I was getting an error as follows.

2019-12-17T19:44:52.167+05:30 [main] WARN FilenoUtil : Native subprocess control requires open access to sun.nio.ch
Pass '--add-opens java.base/sun.nio.ch=org.jruby.dist' or '=org.jruby.core' to enable.
java.lang.IllegalCallerException: sun.nio.ch is not open to module org.jruby.dist
at java.base/java.lang.Module.addOpens(Module.java:763)
at org.jruby.dist/com.headius.backport9.modules.impl.Module9.addOpens(Module9.java:28)
at org.jruby.dist/com.headius.backport9.modules.Modules.addOpens(Modules.java:22)
at org.jruby.dist/org.jruby.util.io.FilenoUtil$ReflectiveAccess.(FilenoUtil.java:140)
at org.jruby.dist/org.jruby.util.io.FilenoUtil.getFilenoUsingReflection(FilenoUtil.java:111)
at org.jruby.dist/org.jruby.util.io.FilenoUtil.filenoFrom(FilenoUtil.java:107)
at org.jruby.dist/org.jruby.util.io.ChannelFD.initFileno(ChannelFD.java:42)
at org.jruby.dist/org.jruby.util.io.ChannelFD.(ChannelFD.java:32)
at org.jruby.dist/org.jruby.RubyIO.(RubyIO.java:174)
at org.jruby.dist/org.jruby.RubyFile.(RubyFile.java:281)
at org.jruby.dist/org.jruby.parser.Parser.parse(Parser.java:99)
at org.jruby.dist/org.jruby.Ruby.parseFileAndGetAST(Ruby.java:2709)
at org.jruby.dist/org.jruby.Ruby.parseFileFromMainAndGetAST(Ruby.java:2702)
at org.jruby.dist/org.jruby.Ruby.parseFileFromMain(Ruby.java:2686)
at org.jruby.dist/org.jruby.Ruby.parseFromMain(Ruby.java:642)
at org.jruby.dist/org.jruby.Ruby.runFromMain(Ruby.java:588)
at org.jruby.dist/org.jruby.Main.doRunFromMain(Main.java:415)
at org.jruby.dist/org.jruby.Main.internalRun(Main.java:307)
at org.jruby.dist/org.jruby.Main.run(Main.java:234)
at org.jruby.dist/org.jruby.Main.main(Main.java:206)
jruby 9.2.7.0 (2.5.3) 2019-04-09 8a269e3 OpenJDK 64-Bit Server VM 11.0.5+10 on 11.0.5+10 +jit [darwin-x86_64]

It didn't stop the printing of the version though.

A bit of googling pointed to the discussion at JRuby Github repo which further pointed to another discussion in jruby-launcher repo. I was not aware of the reason of the existence of jruby-launcher but it is an interesting take on a difficult situation.

From the discussion, it seemed that the fix was available in version 1.1.10. So, I tried looking at my Gemfile.lock to check the version I am using; but there was no version mentioned in that. However, it was present in the list of installed gems. I added the fixed version in the Gemfile and the issue was fixed.

Wednesday, 28 August 2019

Setting up jEnv

With OpenJDK, there is a need to have multiple JDK versions installed as there are many releases that are used by different projects. More and more projects are supporting multiple Java versions as well. This problem was not as prevalent in the Java ecosystem but the Ruby ecosystem had it long back. There are multiple tools to manage Ruby versions. The approach of rbenv is probably the simplest. On the same lines, we have jEnv for Java.

To setup jEnv, the steps are simple

  1. Install jEnv (brew install jenv)
  2. Setup shell profile (~/.bash_profile)
  3. export PATH="$HOME/.jenv/bin:$PATH"
    eval "$(jenv init -)"

  4. Install Java
  5. Set local or shell version of Java (jenv local openjdk64-1.8.0.192)

Thursday, 22 August 2019

Jacoco instrumentation error

In a Java project build, I was getting an error complaining about Jacoco failing to instrument a particular class. Looking at the complete stack trace of the error, it was an exception about index going out of bounds.

Caused by: java.lang.ArrayIndexOutOfBoundsException: 6
at org.jacoco.core.internal.BytecodeVersion.get(BytecodeVersion.java:41)
at org.jacoco.core.instr.Instrumenter.instrument(Instrumenter.java:87)
at org.jacoco.core.instr.Instrumenter.instrument(Instrumenter.java:123)
... 25 more
Fortunately, a little bit of search showed that it is related to Java. My initial version of Java was '1.8.0.121'. The fix was backported to '1.8.0.172'. I also had '1.8.0.192' installed. So, I switched versions and tried a clean build. It worked fine.

Tuesday, 16 April 2019

Learning Puppet: Idempotence

I am learning Puppet so I wanted to try out the manifests on a local VM. So, I created a Vagrant based Arch linux VM.

Vagrant.configure("2") do |config|
 config.vm.box = "archlinux/archlinux"
 config.vm.hostname = "learn.box"
 config.vm.synced_folder "puppet/", "/home/vagrant/puppet"
end


When I tried to bring the VM up, I got the following error.

Vagrant was unable to mount VirtualBox shared folders. This is usually
because the filesystem "vboxsf" is not available. This filesystem is
made available via the VirtualBox Guest Additions and kernel module.
Please verify that these guest additions are properly installed in the
guest. This is not a bug in Vagrant and is usually caused by a faulty
Vagrant box. For context, the command attempted was:

mount -t vboxsf -o uid=1000,gid=1000 vagrant /vagrant

The error output from the command was:

: Invalid argument


It did not stop the VM from coming up so I was able to test. I created the following manifest to start with and it worked fine.

file {
 '/tmp/motd': content => 'hello puppet'
}


I moved on to actually installing a package.

class logstash{
  package{
    "logstash":
      name => "logstash",
      alias => "logstash",
      ensure => "7.0"
  }

}

I tried to apply it using the following:

puppet apply logstash.pp -v

The package was not installed. I soon figured out it was because, I was not applying the class. So, I modified the manifest to the following:

  package{
    "logstash":
      name => "logstash",
      alias => "logstash",
      ensure => "7.0"
  }


Now Puppet tried to install the package but it failed with the following error.

Error: Parameter ensure failed on Package[logstash]: Provider pacman must have features 'versionable' to set 'ensure' to '7.0' (file: /home/vagrant/logstash.pp, line: 2)

Arch linux uses pacman as its package manager and Puppet was trying to install a specific version of the package using that. However, pacman does not support different versions of the same package. So, Puppet can't ensure a specific version. So, I changed the manifest to the following:

  package{
    "logstash":
      name => "logstash",
      alias => "logstash",
      ensure => "installed"
  }

Now, Puppet attempted to install the package again and hit the following error.

Error: Execution of '/usr/bin/pacman --noconfirm --needed --noprogressbar -Sy logstash' returned 1: error: you cannot perform this operation unless you are root.
Error: /Stage[main]/Main/Package[logstash]/ensure: change from 'absent' to 'present' failed: Execution of '/usr/bin/pacman --noconfirm --needed --noprogressbar -Sy logstash' returned 1: error: you cannot perform this operation unless you are root.


So, I attempted using sudo. The process was taking time. I waited for some time then interrupted it and tried to see the logs. The logs seemed fine so I started again and the this time I got the following error.

Error: Execution of '/usr/bin/pacman --noconfirm --needed --noprogressbar -Sy logstash' returned 1: :: Synchronizing package databases...
error: failed to update core (unable to lock database)
error: failed to update extra (unable to lock database)
error: failed to update community (unable to lock database)
error: failed to synchronize all databases
Error: /Stage[main]/Main/Package[logstash]/ensure: change from 'absent' to 'present' failed: Execution of '/usr/bin/pacman --noconfirm --needed --noprogressbar -Sy logstash' returned 1: :: Synchronizing package databases...
error: failed to update core (unable to lock database)
error: failed to update extra (unable to lock database)
error: failed to update community (unable to lock database)
error: failed to synchronize all databases


This is specific to pacman. As I had interrupted the execution, the lock file was not deleted, i.e. pacman had not cleaned up properly. I manually removed the lock file from /var/lib/pacman/db.lck and tried to apply the manifest again. It completed successfully and logstash was installed on the VM. Now, it is established that the script works but it might need manual intervention. In other words, the manifest needs to be idempotent so that manual intervention is minimal.

To achieve that, I modified the manifest as follows.

  package{
    "logstash":
      name => "logstash",
      alias => "logstash",
      ensure => "installed"
  }
  file{
    "/var/lib/pacman/db.lck":
      path => "/var/lib/pacman/db.lck",
      name => "/var/lib/pacman/db.lck",
      ensure => "absent"
  }

To test that the manifest is idempotent, I removed logstash and applied the manifest. Midway, I interrupted the execution and re-applied the manifest. This time re-application succeeded. The logs show that the lock file was created when the manifest was applied the previous time and interrupted. It was deleted when the manifest was applied again.

Info: Applying configuration version '1555432879'
Info: Computing checksum on file /var/lib/pacman/db.lck
Info: /Stage[main]/Main/File[/var/lib/pacman/db.lck]: Filebucketed /var/lib/pacman/db.lck to puppet with sum d41d8cd98f00b204e9800998ecf8427e
Notice: /Stage[main]/Main/File[/var/lib/pacman/db.lck]/ensure: removed
Notice: Applied catalog in 0.25 seconds


Key take-aways:
  1. Puppet is a framework that depends on providers to install packages and its capabilities are as good as the providers.
  2. Without idempotent behaviour, Puppet manifests will not be achieve much of automation.

Monday, 21 January 2019

Finding where a package is installed

Recently, I faced an issue on a CentOS box where I had to find the location of a package. My application was running on it and I wanted to modify its config. I was unable to pick the config in /etc so I figured it must be in the folder in which the application is installed.

The application was installed as a daemon but it seems the PATH was not updated. I searched in the usual locations of binaries and packages but I could not find it. The only option left for me was to find out where it was installed.

Using the following command showed all the files of the application package.

rpm -ql <my application>

From the list, I could see where my config file was placed during install.

Friday, 18 January 2019

Unexpected exit of Elasticsearch container

I was trying the Elasticsearch container image on docker hub. I was disappointed by the complete lack of error message in the following scenario.

I had my setup defined in a compose file. When I tryed to start the container, I got only the following log line.

OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.

As it is a warning, I expected the container to start but it had not started. Digging a bit more, I found that the exit code was 137 which meant the container needed more memory than the docker daemon was configured for. The fix is quite easy of course but I think it could have been communicated better.