Benjamin Smith

I’ve been on a major self improvement kick lately. I’ve kicked several crazy habits including, but not limited to:

• Smoking
• Biting my nails
• Soda
• Sugar (practicing moderation)

Next on my self improvement quest, I decided to start monitoring bandwidth usage and tracking it on my blog so I(and you!) can follow how much I’m using, and try to trim it down to a reasonable level. I’m not going to play around, I am seriously addicted to the internet. It is one of my oldest, hardest to moderate addictions EVER. I figured if I kept track of how much I actually use on a daily basis that it would help me moderate myself.

So, this morning I downloaded and installed vnstat and initialized the database for my single interface like so:

sudo vnstat -u -i eth0

Where eth0 is the interface all my network traffic routes through. To track multiple interfaces, run the above command again, swapping out eth0 for the other interface.

Next I fired up the following cron:

*/5 * * * * if [ -x /usr/bin/vnstat ] && [ `ls /var/lib/vnstat/ | wc -l` -ge 1 ]; then /usr/bin/vnstat -u; fi

It samples usage on any interface you specify every 5 minutes and creates awesome statistics from it. So, whenever I have a quick break between tasks, I run vnstat and get a glimpse at my usage.

Now, this is the part that scared me a bit.. I never really thought about how much I use. I fired all of this up at ~9:00AM, the snapshot below was taken at 10:45AM.

bsmith@bsmith-desktop [10:47:06 (0.17)] ~ 
-> vnstat
Database updated: Tue Jan 26 10:45:01 2010

    eth0

       received:      40.68 MB (81.6%)
    transmitted:       9.89 MB (18.4%)
          total:      50.57 MB

                    rx     |     tx     |  total
    -----------------------+------------+-----------
        today     40.68 MB |    9.89 MB |   50.57 MB
    -----------------------+------------+-----------
    estimated        89 MB |      20 MB |     109 MB

I, um, yeah I use the internet, a lot..

There are lots of other fun things you can do with vnstat, but I’ll mainly just look at incremental snapshots, and the money shot: daily usage.

Daily output looks like this:

bsmith@bsmith-desktop [10:47:41 (0.09)] ~ 
-> vnstat -d

eth0  /  daily

day         rx      |     tx      |  total
------------------------+-------------+----------------------------------------
26.01.     42.19 MB  |   10.35 MB  |   52.53 MB   %%%%%%%%%%%%%%%%%%%%:::::
------------------------+-------------+----------------------------------------
estimated       92 MB  |      22 MB  |     114 MB

As said previously, I’m going to post my bandwidth usage daily, either here, or on my Posterous for the next month. I don’t really have a goal here, just want to be able to trim my usage to hopefully stave off some of my internet “dependency” issues..

Stay Tuned!

I constantly forget that I already have screen sessions available when I log in on Monday mornings. I don’t like that because it’s wasteful, but I’m just lazy enough that the thought alone isn’t enough to make me do something about it until now. So, to combat this, I whipped the script below as a reminder to use one of them, or get rid of them.

This will only fire when you are not already in a screen session, courtesy of the $WINDOW environment variable.

Add this to your ~/.bashrc

function list_screens() {
    if ! test $WINDOW;
    then
        echo
        echo "Current screen sessions available"
        screen -ls
    fi
}

list_screens

Yay! Now I can continue to be lazy!

You can find this and other junk over in my dotfiles at bitbucket: https://bitbucket.org/benjaminws/dotfiles/

At the office I recently upgraded a machine to CentOS 5.2 from CentOS 4.7 (filesystem stayed intact). Everything was cool until I ran into a bit of an issue when attempting to update and regen the yum cache. I got a python traceback that looked like this:

# yum update
base      : ################################################## 2458/2458
update                    100% |=========================|  951 B    00:00
primary.xml.gz            100% |=========================| 242 kB    00:00
Traceback (most recent call last):
  File "/usr/bin/yum", line 29, in ?
    yummain.main(sys.argv[1:])
  File "/usr/share/yum-cli/yummain.py", line 105, in main
    result, resultmsgs = base.doCommands()
  File "/usr/share/yum-cli/cli.py", line 289, in doCommands
    self._getTs()
  File "/usr/lib/python2.4/site-packages/yum/depsolve.py", line 85, in _getTs
    self._getTsInfo()
  File "/usr/lib/python2.4/site-packages/yum/depsolve.py", line 91, in _getTsInfo
    self._tsInfo.setDatabases(self.rpmdb, self.pkgSack)
  File "/usr/lib/python2.4/site-packages/yum/__init__.py", line 537, in <lambda>
    pkgSack = property(fget=lambda self: self._getSacks(),
  File "/usr/lib/python2.4/site-packages/yum/__init__.py", line 392, in _getSacks
    self.repos.populateSack(which=repos)
  File "/usr/lib/python2.4/site-packages/yum/repos.py", line 242, in populateSack
    sack.populate(repo, mdtype, callback, cacheonly)
  File "/usr/lib/python2.4/site-packages/yum/yumRepo.py", line 168, in populate
    dobj = repo_cache_function(xml, csum)
  File "/usr/lib64/python2.4/site-packages/sqlitecachec.py", line 42, in getPrimary
    self.repoid))
TypeError: Can not prepare packages insertion: table packages has no column named location_base

Eeek! Got the same error when running yum makecache. Quick look at the traceback is somewhat obvious, the DB schema changed, but the DB didn’t get upgraded! Easy fix is to clear out your yum cache like so:

# cd /var/cache/yum
# rm -r *

And then you’ll want to regenerate your cache like so:

# yum makecache

Hope this helps anyone else who runs into this issue!

Since upgrading to Ubuntu 8.10 (Intrepid) I had been having trouble with my SSL VPN connectivity. Rather than sit down and figure it out I chose to fire up a virtual box instance of 8.04 and use it to get work done. Finally, tonight, I hit the tipping point and decided to figure this out. First I discovered debug mode with the SSL client. You can enable it on the command line by running

# startct -d

That will put a log in /var/log/AvConnect.log with lots of debugging goodness. The first thing I noticed was a fun error that looked kinda like this:

XGSSL::Connect() DoHandShake Failed!!!

Obviously enough, this led me to believe that this was a problem with the ssl handshake.

So, I decided at this point to inspect the guts of the Aventail client. Seemed simple enough, a few shell scripts (startct,stopct,uninstall) and some perl (cctel.pl) and then there’s this mysterious binary, AvConnect. I decided to look deeper and found this..

# file /usr/local/Aventail/AvConnect 
/usr/local/Aventail/AvConnect: setuid ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.2.5, dynamically linked (uses shared libs), stripped

Looks like it uses shared libs, let’s see where it’s linking..

# ldd /usr/local/Aventail/AvConnect 
linux-gate.so.1 =>  (0xb8008000)
libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xb7fd4000)
libssl.so.0.9.7 => /usr/lib/libssl.so.0.9.7 (0xb7e93000)
libcrypto.so.0.9.7 => /usr/lib/libcrypto.so.0.9.7 (0xb7d3c000)
libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb7d16000)
libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7bb8000)
/lib/ld-linux.so.2 (0xb8009000)
libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7bb4000)
libz.so.1 => /usr/lib/libz.so.1 (0xb7b9e000)

Did a bit more digging by running some tests with OpenSSL.

# openssl version
OpenSSL 0.9.8g 19 Oct 2007

# openssl s_client -CAfile /etc/ssl/certs/Entrust_Root_Certification_Authority.pem -connect <myvpnhost>:443 -showcerts

Which seemed to work fine..

So, puzzled I compared the openssl package on my 8.04 VM to what I had installed, shockingly they matched.

# openssl version
OpenSSL 0.9.8g 19 Oct 2007

Taking a shot in the dark I decided to upgrade my local install of openssl from source. Fairly run of the mill install..

# wget http://openssl.org/source/openssl-0.9.8i.tar.gz

# tar zxvfm openssl-0.9.8i.tar.gz

To get the shared objects, you’ll want to run configure like so:

# cd openssl-0.9.8i
# ./config shared

Now, do the magic..

# make && make test

Install it in the default location /usr/local/ssl

# sudo make install

Then, you’ll want to update your links in /usr/lib to point at the new shared objects.

# sudo ln -s /usr/local/ssl/lib/libcrypto.so /usr/lib/libcrypto.so

# sudo ln -s /usr/local/ssl/lib/libssl.so /usr/lib/libssl.so

# sudo rm /usr/lib/libcrypto.so.0.9.7

# sudo ln -s /usr/lib/libcrypto.so /usr/lib/libcrypto.so.0.9.7

# sudo rm /usr/lib/libssl.so.0.9.7

# sudo ln -s /usr/lib/libssl.so /usr/lib/libssl.so.0.9.7

Then test it out by running startct and you should be able to connect as usual. I have yet to figure out why the ssl handshake fails with the packaged version of openssl, so if anyone has any clue about that or has a smarter work around, please let me know :)

This is, of course, not the recommended action for people new to Linux/Ubuntu and I provide no guarantee that it will work for you. I just wished to provide a possible work around for anyone having the same issue googling around aimlessly.

Enjoy

EDIT: More comprehensive shuffling of lib links per comments and observations below. Thanks devodl and js!

I’ve recently decided to evaluate other shell’s and ran across one that looked interesting: fish.

You can read more about fish: here and here

Based on what I’ve seen I’ve decided to try it out. Before I dive in too deep with fish, does anyone have any opinions, caveats, issues to mention about it?

If I don’t like it I’ll keep trying out zsh but most likely end up back in bash :)

I’d love to go into detail about how I hacked together this awesome way to install wubi from the loopback mount onto it’s own partition. Truth be told, save for a quick boot loader edit, I did almost nothing.

Take a look at this site: http://lubi.sourceforge.net/lvpm.html for a complete, visual step-by-step on how to install your wubi instance directly to your hard drive.

The only issue I ran into was with booting into Parted Magic. The DEB I was instructed to install pointed at the wrong disk for root() in the GRUB line. It was as easy as editing /boot/grub/menu.lst to point it at the right place.

In /boot/grub/menu.lst search for ubninit. Change the root line above where you landed to look like this:

root(hd0,)

Rather than this:

root(hd1,)

Enjoy!

Today I had to yank a line from my ~/.ssh/known_hosts file on my jump box here at the office and I noticed that I had a lot of known hosts! I’d be interested to see how many you have.

My work jump box:

bsmith@ops654 [09:06:04 (0.00)] ~/.ssh 
-> wc -l known_hosts 
1155 known_hosts

My desktop:

AGCOM\bsmith@bsmith-laptop [09:07:21 (0.22)] ~/.ssh 
-> wc -l known_hosts 
65 known_hosts

Here is a fun command we use at the office that makes extracting hardware info from a remote machine easy as pie. It’s called dmidecode and it can be quite useful. Here is one section of the information it can provide, extracted from one of my servers(some bits removed):

 Handle 0x0100
    DMI type 1, 27 bytes.
    System Information
            Manufacturer: Dell Inc.
            Product Name: PowerEdge 1950
            Version: Not Specified
            Serial Number: XXXXXX
            UUID: XXXXXXXXXXXXXXXXXXXX
            Wake-up Type: Power Switch

You are, unfortunately, at the mercy of the hardware manufacturer for accuracy of the hardware information, but thus far I’ve found it to be accurate.

Enjoy.

I recently decided that I needed to come up with a system for managing custom and commonly used python modules. In doing so I decided upon a directory structure similar to this:

volatile/
dev/
prod/

Where volatile would be an unsafe set of packages, not yet tested in a staging environment (constantly updated), dev would be the packages currently in the staging environment and prod would be the packages currently installed, or ready to deploy to the production environment. This has been done before, nothing new. Along the way I thought to myself, how can I conserve disk space, avoiding up to three copies of the same file? An easy and logical way to avoid this would be hard links. So, rather than having three copies of a file that gets approved through to production, I would hardlink back to the original. I will demand that new files into volatile be uniquely named, so avoid overwriting. As files become obsolete, they will be pruned down the line (a system to handle this would need to be developed).

SO! Now that we got passed the background, too the yummy tips.

I have this awesome directory structure of existing files, right? How do I duplicate it creating links instead of files, and creating the directories as needed(can’t safely hardlink directories). I knew I could use rsync with --link-dest. It would look something like this:

rsync -avz --link-dest /volatile /dev/ \
rsync -avz --link-dest /volatile /prod/

-a says to preserve things, -v is verbose, -z says compress and --link-dest says to make links of the destination files.

I also thought of some find/exec magic creating directories and linking files, it would look something like this:

cd /volatile; find . -type d -exec mkdir -p /dev/{} \; \
find . type f -exec ln {} /dev/{} \;

Repeating the same thing for the /prod/ directory.

Both the above solutions feel hackish, but would work. I thought, there has to be a simpler way, this is a fairly routine operation. After checking out google, I was pointed to the cp(1) man page, which uncovered this gem:

-l, --link
       link files instead of copying

Woah, I knew it. So much shorter and uses the right tool for the job!

cp -r -l volatile/* dev/; cp -r -l volatile/* prod/

It’s amazing what you’ll uncover with a tad bit of googling and man page searching!

Nothing really exciting to report on day 3. I'm pretty sure I got this licked. I did discover that our Cisco VPN tunnel for clients was borked with the last maintenance on the network, so I lost my backdoor to be able to work from home via Linux :(.

Day 4 I found a reliable way to crash the Evolution "exchange backend". I get mails from Mailman mailing lists as they come in, rather than a daily digest. Clicking on one with the preview pane on will break the "exchange backend" causing me to have to restart the client. This happens after browsing around in "non-list" mails.

Also on day 4 I found myself needing a proprietary piece of client software to access a file server that I manage. I was slightly discouraged until I realized it was written in Java. I then discovered that they don't provide the source, but they do provide the jar files, and an RPM. I then used alien to convert the rpm to a deb, and I was well on my way to managing that file server.

I'm loving this!