Saturday, 22 January 2011

Setting up a software project

Recently, I have started a software project with a few of my friends. I shall speak about how I set up communication media for the project. We started with creating a secret group in facebook. That way we are able to discuss the project even when we are at our homes. Facebook also has functionalities of creating documents for the group. However, facebook chat is not quite reliable as a group chat and it does not support voice. [It reminds me of good old yahoo chat days. :) ] Document creation feature is in no way an alternative to attachments and code sharing. So, the next step was to create a mailing list.

With a mailing list, now we can have interleaved replies, no need to refer to the line we are replying to. Moreover, with a mailing list we can share documents easily as attachments and everyone can look into the other's progress at his/her convenience. For group chat, we started an IRC channel on freenode and used skype for voice chat in group.

Now, we were all set for discussing any ideas we had at any time. We only lacked in a tool for sharing code efficiently. We opted for a code hosting site. I preferred a distributed version control system over a centralised one because we expect a lot of code merges. Moreover, due to fluctuating internet, having the ability to commit locally would be an advantage. So, we opted for mercurial(hg) hosting at www.bitbucket.org. Mercurial is a mature distributed version control system and bitbucket has a feature rich user interface. Now, we are all set for the project and hope it turns successful.

Monday, 3 January 2011

Search Engine Blacklisting extension

Recently, Jason Chen released an Chrome extension to blacklist spam websites in search engine results.
The extension is under active development and is for testers now. As far as I remember, there have been tools for this by McAfee and AVG. However, I believe that such blacklisting should be done at the server side. It is funny sometimes how search engines like Google have fake sites topping their results.

Wednesday, 29 December 2010

Hibernate from E17 on Arch linux

Earlier this year, I had posted about modifying {install_folder}/etc/enlightenment/sysactions.conf to obtain "Sleep" feature from Enlightenment on Arch linux. I did not mention about hibernation because I had not got it to work. Hibernation can be enabled likewise modifying sysactions.conf to replace
action:   hibernate /etc/acpi/hibernate.sh force
by
action:   hibernate /usr/sbin/pm-hibernate
However, to enable hibernation I had to add "resume" to HOOKS variable defined in /etc/mkinitcpio.conf and add resume=/path/to/swap to my kernel line in /boot/grub/menu.lst.

Sunday, 26 December 2010

Sharing internet connection through Wlan

I wanted to share my internet connection with my roomies using wireless LAN. I had done it using bridging in Windows; but I hardly use Windows. So, I needed a solution in linux. I started discussing about it on IRC. At #gentoo, I met a French guy, named Francis Galiegue, who gave me an elegant solution.

We have three laptops and the connection is through wired ethernet. So, the solution was to use one laptop (mine) as a wireless access point giving access to the internet. In this setup, it is granted that the laptop has network connectivity to the internet, and that it has a WiFi device (internal or external) recognised by linux and the kernel is recent enough (2.6.27 or later is recommended). Once you meet the prequisites, there are four parts, which are solved by four linux daemons elegantly:

0. without even configuring the WiFi device, check that the laptop can connect to the internet
1. setting up basic iptables rules (see rule set 1);
2. cooking up a set of rules so that the laptop can access the Internet via the appropriate device (see rule set 2);
3. configure a DHCP server (using dhcpd), complete the firewall rule set to allow it to work (see rule set 3);
4. configure a name server (using BIND), complete the firewall rule set to allow it to work (see rule set 4);
5. configure an access point (using hostapd) - and no, no firewall rules are necessary for the access point to operate (iptables operates at the network layer, hostapd operates below that level);
6. complete firewall configuration so that "client" computers (the other laptops) can actually connect to the Internet.


Rule set #1:

The goal here is to create a generic table which uses Linux's netfilter connection tracking abilities. Here we use the "state" module, which recognizes four states:
  1. ESTABLISHED: the incoming packet is part of a connection known toLinux' connection tracking;
  2. RELATED: the incoming packet either directly relates to, or establishes a new connection related to, a connection known to Linux's connection tracking - such packets are of two types:
    1. ICMP messages (such as: "no route to host", "access prohibited", others);
    2. connection triggers from builtin modules (such as FTP data connections, others);
  3. INVALID: the incoming packet has an invalid payload (header length and/or checksum mismatch at the network layer or upper);
  4. NEW: the incoming packet tries to initiate a new connection.
We create a new chain, named "connstate" (ie, "connection state"), attached to the "filter" table. The purpose of this chain will be to handle all four connection states known to the "state" module. Eventually, all packets, either incoming (INPUT), outgoing (OUTPUT) or going through (FORWARD) will go through this chain, except for the loopback interface (lo), which is special:


#
# Create the chain - note that by default, if the table (the -t option of
# iptables) is not specified, the default is filter - this is what we want
#
iptables -N connstate
#
# All packets of connections already known to netfilter's state tracking
# (ESTABLISHED) or directly related (RELATED) should pass
#
iptables -A connstate -m state --state ESTABLISHED,RELATED -j ACCEPT
#
# All packets deemed invalid by netfilter should be dropped
#
iptables -A connstate -m state --state INVALID -j DROP
#
# From then on, packets have to be NEW. One thing: if the packet is TCP and does
# not have the SYN bit set (which it should have, see RFC 793) should be
# dropped...
#
iptables -A connstate -m state --state NEW -p tcp ! --syn -j DROP
#
# Any other NEW packets are returned to the caller
#
iptables -A connstate -m state --state NEW -j RETURN
#
# Normally, no packet ever should reach this point, netfilter must/will have
# sorted them out earlier on. If not, this is clearly a bug, so log them at the
# highest log level avaibale (CRIT == critical), and drop them for safety.
#
iptables -A connstate -j LOG --log-level CRIT --log-prefix "CONNSTATE BARF: "
iptables -A connstate -j DROP
#
# There is one exception to the rules above: the loopback interface. Packets
# going through the loopback will not go through the normal chain processing,
# we need to accept them unconditionally at the input and output phase.
#
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT
#
# From here on, the first thing to do is to make all packets of all builtin
# chains of the filter table go through this chain.
#
for i in INPUT OUTPUT FORWARD; do iptables -A $i -j connstate;done

At this stage, this chain enforces no restriction on incoming or outgoing traffic, except for two quite important things:
  1. no TCP and/or IP header fragmentation attack is possible anymore: as soon as you use connection tracking (as we do here), the firewalling engine must have all protocol headers to decide how to deal with the whole packet (attempted fragmentation attacks will be deemed INVALID and therefore DROPped);
  2. some stack implementations disobey RFC 793 with respect to TCP connection initiation, since they don't set the SYN bit on the initial TCP packet of the connection: these will be dropped as well (observed with some versions of Windows).

Rule set #2:

We proceed to allow ICMP (ping) traffic from the wifi interface.


iptables -N local_to_eth0
iptables -A local_to_eth0 -j ACCEPT
iptables -A OUTPUT -o eth0 -j local_to_eth0
iptables -N ping
iptables -A ping -p icmp --icmp-type echo-request -m limit --limit 2/sec -j ACCEPT
iptables -A ping -p icmp --icmp-type echo-request -m limit --limit 2/sec -j DROP
iptables -N eth0_to_local
iptables -A eth0_to_local -j ping
iptables -A INPUT -i eth0 -j eth0_to_local
for i in INPUT OUTPUT FORWARD; do iptables -P $i DROP;done


Rule set #3:

We proceed to allow ICMP (ping) traffic from the wifi interface.


iptables -N dhcp
iptables -A dhcp -p udp --dport 67:68 -j ACCEPT
iptables -N wlan0_to_local
iptables -A wlan0_to_local -j ping
iptables -A wlan0_to_local -j dhcp
iptables -A INPUT -i wlan0 -j wlan0_to_local


There are two ways of being a gateway:

  1. configure a dhcp server and bind
  2. use dnsmasq

We are using the former method here. So, you might want to query your package manager for dhcpd and bind to see whether they are installed.

Next, find out your domain name (hostname -f). Let us say your domain name is "domain_name". Now pick a hostname for your system, say "hostname.domain_name". You might opt for a two component domain name. Now, proceed to assign the selected hostname to your system. Pick an IP in RFC1918 range to assign to this name, say 192.168.1.4. Edit /etc/hosts and add the following line:


192.168.1.4 hostname.domain_name hostname


Now let us ensure that the hostname is assigned to the machine at boot time. It can be done by editting /etc/conf.d/net and /etc/conf.d/hostname in Gentoo linux or by editting /etc/rc.conf in Arch linux. (Set it to the full qualified hostname, i.e. "hostname.domain_name" not just "hostname".)

Next, we setup wlan0 with the address 192.168.1.4 and a /24 subnet mask. It can be done using ifconfig as follows:

ifconfig wlan0 192.168.1.4 netmask 255.255.255.0

This can also be put into /etc/rc.conf so that it is done each time during boot. You may wish to cross check the IP of the wifi device. (See ifconfig ouput and try to ping the IP.)

Now, we configure the DNS server daemon, named. First of all, we are going to create two zone files: one for "domain_name" and the other for 1.168.192.in-addr.arpa. The file /var/named/pri/domain_name.zone is as follows:

$TTL 1d
@       IN      SOA     hostname.domain_name. you.email.address.here.  (
                                      2010102401 ; Serial
                                      28800      ; Refresh
                                      14400      ; Retry
                                      3600000    ; Expire
                                      86400 )    ; Minimum
              IN      NS      fool.man.machine.

5 IN PTR hostname.domain_name.


Now, we edit the named.conf as follows:


//
// /etc/named.conf
//

acl "trusted" {
127.0.0.0/8;
        ::1/128;
        10.142.81.0/24;
};

options {
directory "/var/named";
pid-file "/var/run/named/named.pid";
auth-nxdomain yes;
datasize default;
// Uncomment these to enable IPv6 connections support
// IPv4 will still work:
// listen-on-v6 { any; };
// Add this for no IPv4:
// listen-on { none; };
listen-on {
127.0.0.1;
192.168.1.4;
};


// Default security settings.
allow-query {
trusted;
};
allow-recursion { 127.0.0.1; };
allow-transfer { none; };
allow-update { none; };
version none;
hostname none;
server-id none;

// FORWARDING
forward first;
forwarders {
// The service provider's DNS first
<nameserver>;
<nameserver>;
<nameserver>;
<nameserver>;
                4.2.2.1;                // Level3 Public DNS
                4.2.2.2;                // Level3 Public DNS
                8.8.8.8; // Google Open DNS
                8.8.4.4; // Google Open DNS
};
};

view "internal" in {
match-clients { trusted; };
recursion yes;
additional-from-auth yes;
additional-from-cache yes;

zone "localhost" IN {
type master;
file "localhost.zone";
allow-transfer { any; };
};

zone "0.0.127.in-addr.arpa" IN {
type master;
file "127.0.0.zone";
allow-transfer { any; };
};

zone "." IN {
type hint;
file "root.hint";
};

zone "domain_name" {
type master;
file "pri/domain_name.zone";
allow-update {
none;
};
notify no;
};

zone "1.168.192.in-addr.arpa" {
type master;
file "pri/1.168.192.in-addr.arpa.zone";
allow-update {
none;
};
notify no;
};
};

logging {
        channel xfer-log {
                file "/var/log/named.log";
                print-category yes;
                print-severity yes;
                print-time yes;
                severity info;
        };
        category xfer-in { xfer-log; };
        category xfer-out { xfer-log; };
        category notify { xfer-log; };
};

Now, lets start the server. On Arch, I do it using the following command.

/etc/rc.d/named start

We then edit /etc/resolv.conf.head to add the following line

search domain_name

and /etc/resolv.conf.tail to add the following line.

nameserver 127.0.0.1

Now, the nameserver can be tested using commands like the following.

host hostname.domain_name 127.0.0.1
host slashdot.org 127.0.0.1


You might like to add named to the list of daemons to be started at boot time. I prefer starting them each time.

Rule set #4:

# create a new chain
iptables -N local_to_wlan0
# the only rule of this chain is to accept
iptables -A local_to_wlan0 -j ACCEPT
# In the OUTPUT chain, every packet going out by wlan0 interface is branched out to
# local_to_wlan0 and as a result everything out to wlan0 is accepted.
iptables -A OUTPUT -o wlan0 -j local_to_wlan0


The following iptables rules are to allow the other machines in the LAN to access the DNS server.
Bind listens to TCP/53 and UDP/53 and thus traffic on those ports is accepted.

Rule set #5:

iptables -N named
iptables -A named -p udp --dport 53 -j ACCEPT
iptables -A named -p tcp --dport 53 -j ACCEPT
iptables -A wlan0_to_local -j named


We proceed to configure the dhcp server. The configuration in /etc/dhcpd.conf is as follows:

#
# We don't want dynamic DNS here
#
ddns-update-style none;
subnet 192.168.1.0 netmask 255.255.255.0 {
authoritative;
option subnet-mask 255.255.255.0;
option domain-name "domain_name";
option domain-name-servers 192.168.1.4;
option routers 192.168.1.4;

pool {
range 192.168.1.2 192.168.1.254;
allow unknown-clients;
}
}


Then we have a final set of rules to connect the two interfaces.

Rule set #6:

iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
iptables -N wlan0_to_eth0
iptables -A wlan0_to_eth0 -j ACCEPT
iptables -A FORWARD -i wlan0 -o eth0 -j wlan0_to_eth0


The last piece is hostapd configuration. It is given as follows:

interface=wlan0
driver=nl80211
logger_syslog=-1
logger_syslog_level=0
logger_stdout=-1
logger_stdout_level=0
dump_file=/tmp/hostapd.dump
ctrl_interface=/var/run/hostapd
ctrl_interface_group=0
ssid=test
hw_mode=a
channel=60
beacon_int=100
dtim_period=2
max_num_sta=255
rts_threshold=2347
fragm_threshold=2346
macaddr_acl=0
auth_algs=3
ignore_broadcast_ssid=0
wmm_enabled=1
wmm_ac_bk_cwmin=4
wmm_ac_bk_cwmax=10
wmm_ac_bk_aifs=7
wmm_ac_bk_txop_limit=0
wmm_ac_bk_acm=0
wmm_ac_be_aifs=3
wmm_ac_be_cwmin=4
wmm_ac_be_cwmax=10
wmm_ac_be_txop_limit=0
wmm_ac_be_acm=0
wmm_ac_vi_aifs=2
wmm_ac_vi_cwmin=3
wmm_ac_vi_cwmax=4
wmm_ac_vi_txop_limit=94
wmm_ac_vi_acm=0
wmm_ac_vo_aifs=2
wmm_ac_vo_cwmin=2
wmm_ac_vo_cwmax=3
wmm_ac_vo_txop_limit=47
wmm_ac_vo_acm=0
eap_server=0
own_ip_addr=127.0.0.1
wpa=1
wpa_passphrase=<your passphrase>

Each time I can start sharing using the following commands.

ifconfig wlan0 192.168.1.4 netmask 255.255.255.0
/etc/rc.d/iptables start
/etc/rc.d/hostapd start
/etc/rc.d/dhcpd start
/etc/rc.d/named start


For a rather detailed reading, check out this webpage.

Saturday, 25 December 2010

Sent mail appears in Thunderbird's Inbox

When retrieving mail using Thunderbird, I noticed that all mails that I have sent using the web interface are also downloaded but they are put into the Inbox folder. Obviously, I would like Thunderbird to put them into "Sent" folder. However, over multiple versions this did not change; but the solution is pretty simple. You just have to create a filter redirecting mails sent from your email to the "Sent" folder.
Tools > Message Filters
Redirect sent mail to "Sent" folder

Saturday, 27 November 2010

Arch Linux review

I believe it is time I can provide a substantial review of Arch linux distribution. I have already written about my switch to Arch linux; so I will not reiterate those ideas here. I will just attempt to clarify your view towards Arch linux because most distributions are merely reflections of viewpoints of the developers of the distribution.

Arch linux is a rolling release distribution that provides bleeding edge software. That is all to it. Now, with bleeding edge software, there are always stability issues. If you are looking for rock-solid stability, go for Debian without any further thought. However, do consider the fact that if software is bleeding edge then most likely the bugs are upstream than with the distribution. This notion is reinforced by the fact that Arch does not mod packages. Arch branding is optional. When I started using Arch, I had reported a firefox issue which turned out to be an upstream problem. This was within 24 hours. Then there was the valgrind issue which I did talk about at length. So, basically Arch people respond fast. However, the IRC channel #archlinux is not very friendly (may be I still have a hang over from #gentoo). Arch is good at performance; boots up fast. It has decent support for hardware. Installing drivers for my WLAN was not at all difficult.

On the downside, some aspects that are not addressed by upstream properly are not handled by Arch appropriately. [Many distributions actively patch packages before release; though upstream commitment of the patches is a different scenario.] It is basically Arch policy. I have two unsolved bugs about the Arch kernel: one for 2.6.34 and one for 2.6.35. Also, my SD-card reader was not working properly before 2.6.35.

Yesterday, I upgraded to 2.6.36 and there were two error messages at boot time. One was about tomoyo-init scripts and the other was about HDA-Intel being unknown hardware. The former is harmless and the later was easy to resolve using the wiki. There still remain two things about Arch that annoy me:
Yet again, they are mostly upstream issues. Thus, Arch linux is a nice distribution; but it may not be the best choice for new users.

Tuesday, 9 November 2010

Cache_clean to clean up older packages

I have a 10 gig / partition and I had only about a gig left on it. So, I decided to clean up. I was aware that pacman stores multiple versions of packages so I checked out the size of /var/cache/pacman/pkg and it tuned out to be 4.9 gigs, i.e. more than 50% of my used space. The way to regain disk space was clear. I started looking up ways to clean up and found a python script in AUR. A snippet of the script's preview run is below.

['mercurial-1.5.1-1-x86_64.pkg.tar.xz', 'mercurial-1.5.2-1-x86_64.pkg.tar.xz', 'mercurial-1.5.3-1-x86_64.pkg.tar.xz', 'mercurial-1.5.4-1-x86_64.pkg.tar.xz', 'mercurial-1.6-1-x86_64.pkg.tar.xz', 'mercurial-1.6.2-1-x86_64.pkg.tar.xz', 'mercurial-1.6.3-1-x86_64.pkg.tar.xz']
['xulrunner-1.9.2.2-2-x86_64.pkg.tar.xz', 'xulrunner-1.9.2.3-1-x86_64.pkg.tar.xz', 'xulrunner-1.9.2.4-1-x86_64.pkg.tar.xz', 'xulrunner-1.9.2.6-1-x86_64.pkg.tar.xz', 'xulrunner-1.9.2.7-1-x86_64.pkg.tar.xz', 'xulrunner-1.9.2.8-1-x86_64.pkg.tar.xz', 'xulrunner-1.9.2.9-1-x86_64.pkg.tar.xz', 'xulrunner-1.9.2.10-1-x86_64.pkg.tar.xz']


Using the script I cleaned 2.5 gb within a minute. Elegant.
[N.B. - An alternative script in Go is available at github.]

Wednesday, 20 October 2010

Proxy problem with pacman

Recently, I was accessing internet from behind a firewall so I had set http and ftp proxy environment variables. When I ran

pacman -Syu

it could download the database updates for the system packages; but could not retrieve the packages and hung up. I found that wget was able to use the proxy fine. So, I tried

pacman --debug -S mkvtoolnix

to see what was happening and found that it was stuck after detecting the proxy. As wget was working, I tried uncommenting the following line in pacman.conf.

#XferCommand = /usr/bin/wget --passive-ftp -c -O %o %u

This time the upgrade was smooth and I upgraded 66 packages.

Tuesday, 19 October 2010

Sleep from E17 on Arch linux

Recently, I found that sleep and hibernate do not work from E17 while they were working fine from KDE. So, I decided to dig. I found that I had to configure them in sysactions.conf. I had installed e17 in /opt so I searched for it there and found it in /opt/e17/etc/enlightenment. I changed the following line
action:   suspend   /etc/acpi/sleep.sh force
to
action:   suspend   /usr/sbin/pm-suspend
as my system had no such script as sleep.sh and sleep worked fine ine E17.