Is there something I’m missing here?

I ran yum groupinstall "Development Tools", it went well. Because I don’t trust software, I decided to verify with yum grouplist and was surprised to find “Development Tools” missing from the installed list. My lack of trust has been further validated :(.

Is yum trolling me?!

# yum grouplist
Loaded plugins: fastestmirror
Setting up Group Process
Loading mirror speeds from cached hostfile
 * agcom: cf
 * base: cf
 * epel: cf
 * extras: cf
 * update: cf
Installed Groups:
   Dialup Networking Support
   Mail Server
   Network Servers
   Printing Support
   System Tools
   Yum Utilities
Available Groups:
   Administration Tools
   Horde
   Java
   Java Development
   KDE (K Desktop Environment)
   KDE Software Development
   KVM
   Legacy Network Server
   Legacy Software Development
   Legacy Software Support
   Mono
   MySQL Database
   News Server
   Office/Productivity
   OpenFabrics Enterprise Distribution
   PostgreSQL Database
   Ruby
   Server Configuration Tools
   Sound and Video
   Text-based Internet
   Tomboy
   Virtualization
   Web Development
   Web Server
   Window Managers
   Windows File Server
   X Software Development
   X Window System
   XFCE-4.4
Done

FFFFFFFUUUUUUUUUUUU

# yum groupinstall "Development Tools"
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * agcom: cf
 * base: cf
 * epel: cf
 * extras: cf
 * update: cf
Setting up Group Process
Checking for new repos for mirrors
Package flex-2.5.4a-41.fc6.x86_64 already installed and latest version
Package gcc-4.1.2-48.el5.x86_64 already installed and latest version
Package redhat-rpm-config-8.0.45-32.el5.centos.noarch already installed and latest version
Package strace-4.5.18-5.el5_5.5.x86_64 already installed and latest version
Package rpm-build-4.4.2.3-20.el5_5.1.x86_64 already installed and latest version
Package 1:make-3.81-3.el5.x86_64 already installed and latest version
Package 1:pkgconfig-0.21-2.el5.x86_64 already installed and latest version
Package gettext-0.14.6-4.el5.x86_64 already installed and latest version
Package gettext-0.14.6-4.el5.i386 already installed and latest version
Package automake-1.9.6-2.3.el5.noarch already installed and latest version
Package gdb-7.0.1-23.el5_5.2.x86_64 already installed and latest version
Package bison-2.3-2.1.x86_64 already installed and latest version
Package libtool-1.5.22-7.el5_4.x86_64 already installed and latest version
Package autoconf-2.59-12.noarch already installed and latest version
Package gcc-c++-4.1.2-48.el5.x86_64 already installed and latest version
Package binutils-2.17.50.0.6-14.el5.x86_64 already installed and latest version
Package ltrace-0.5-13.45svn.el5.x86_64 already installed and latest version
Package diffstat-1.41-1.2.3.el5.x86_64 already installed and latest version
Package splint-3.1.1-16.el5.x86_64 already installed and latest version
Package 1:valgrind-3.5.0-1.el5.x86_64 already installed and latest version
Package 1:valgrind-3.5.0-1.el5.i386 already installed and latest version
Package systemtap-1.1-3.el5_5.3.x86_64 already installed and latest version
Package subversion-1.4.6-0.1.el5.rf.x86_64 already installed and latest version
Package subversion-1.4.2-4.el5_3.1.i386 already installed and latest version
Package texinfo-4.8-14.el5.x86_64 already installed and latest version
Package python-ldap-2.2.0-2.1.x86_64 already installed and latest version
Package patchutils-0.2.31-2.2.2.x86_64 already installed and latest version
Package byacc-1.9-29.2.2.x86_64 already installed and latest version
Package gcc-gfortran-4.1.2-48.el5.x86_64 already installed and latest version
Package elfutils-0.137-3.el5.x86_64 already installed and latest version
Package rcs-5.7-30.1.x86_64 already installed and latest version
Package automake16-1.6.3-8.el5.1.noarch already installed and latest version
Package automake17-1.7.9-7.el5.2.noarch already installed and latest version
Package automake14-1.4p6-13.el5.1.noarch already installed and latest version
Package automake15-1.5-16.el5.2.noarch already installed and latest version
Package ctags-5.6-1.1.x86_64 already installed and latest version
Package dogtail-0.6.1-3.el5.noarch already installed and latest version
Package swig-1.3.29-2.el5.x86_64 already installed and latest version
Package dev86-0.16.17-2.2.x86_64 already installed and latest version
Package 1:doxygen-1.4.7-1.1.x86_64 already installed and latest version
Package indent-2.2.9-14.fc6.x86_64 already installed and latest version
Package cvs-1.11.22-7.el5.x86_64 already installed and latest version
Package pstack-1.2-7.2.2.x86_64 already installed and latest version
Package oprofile-0.9.4-15.el5.x86_64 already installed and latest version
Package cscope-15.5-15.1.el5_3.1.x86_64 already installed and latest version
No packages in any requested group available to install or update

This evening I found out that either someone hacked my Facebook, or my phone network routed me through Tampa, FL (seen something similar before, just not so far away).  Regardless, I have to say that Facebook seems to have fairly decent security mechanisms.  

That being said, I can't stress enough how important it is to make sure you're as secure as you can be Facebook, so here is how to turn SSL on all-the-effin-time (except with some apps/games that don't support it, *le sigh*).

• Visit: https://www.facebook.com/editaccount.php
• Next to 'Account Security', click 'change'.
• Under 'Secure Browsing (https)', click the 'Browse Facebook on a secure connection (https) whenever possible' check box.
• Now click 'Save'.

There, doesn't that feel better?

To be honest, it was probably my weak ass password (for shame!), and not browsing over a non SSL connection that got my account compromised, but this important and you shouldn't hesitate to do it.

Hopefully by the time you read this sentence, you will have enabled SSL.

Seriously.  Do it.

Intro

OK, so I suck at this.  I set out to write at least two, maybe three blog posts last night.  One of the other two is going to require much more thought and editing, the third is about vaporware (I literally have one test done) and this one, well, this one is the easiest because most of the work is already done ;).

Pretty simple what's going on here, but lets step through it anyway.

• Setup arguments to pass to the constructor
• Create the object
• Setup some options for subscribing to the destination
• Connect to the broker
• Subscribe to events
  • When 'connected' is fired, subscribe to the desired destination
  • When 'message' is fired, ack the message (you could do fancy things with the message here)
  • When 'error' is fired, inform of the error and disconnect
  • When 'SIGINT' is caught by the process, report a tally of work done and disconnect

This will listen for messages until we kill it, or if an error is encountered.  It's single purpose is to consume all the messages and acknowledge them with the broker.

Now we can't just let that consumer sit there being all lazy, lets send it some messages.

A stomp-js producer:

Stepping through this briefly.

• Get number of messages to produce from the arguments
• Decided if we want to get a receipt for messages produced
• Setup arguments for the constructor
• Create the object
• Set the destination for messages
• Connect to the broker
• Subscribe to events
  • When 'connected' is fired, loop 'n' times, sending a message to the broker on each iteration.  Disconnect when complete.
  • When 'receipt' is fired, report the receipt ID
  • When 'error' is fired, inform of the error and disconnect
  • When 'SIGINT' is caught by the process, report a tally of work done and disconnect

Pretty simple stuff!

I'd like to point out that stomp-js does implement the basic transaction support defined by STOMP 1.0.  Below is some example code showing how to implement this.

Basically you call client.begin(), which returns a transaction id.  Then add a 'transaction' header to the message you're sending with the transaction id as the value.  Once the send is done, call client.commit(transaction_id).

Here is the full example:

Fin

That's basically it!

Want to help make stomp-js better?  Have an issue with it?  Want to keep up with progress?  Follow/Fork/Report on the github page! https://github.com/benjaminws/stomp-js

Gary, With Beer: http://frompythonimportpodcast.com/episode-007-gary-with-beer

Living in the Cleveland area, there aren't a lot of "fun" places to drive.  By "fun", I mean low traffic, twisty roads.  We do, however, have the metroparks.  That's as close as I can get to the mountain roads I'm used to back home.

So, today I took a bit of time to map my route home through the metroparks:

View The Fun Way Home in a larger map

If you're familiar with the area, and can think of a better way, make me a map!

Seriously.

If you don’t know, Chris Miller, Dave Stanek and Mike Crute started a podcast some months ago, aptly named from python import podcast. If you do know, great! Keep listening and supporting!

What you might not know is that I crashed the party to babble with the fellas about all things python. Hope they like me, cause I’m sticking around!

In all seriousness, on the most recent episode we talk about PyOhio, the conference process, the various awesome ideas and projects that spawned off of it and lots of other stuff. Definitely worth noting that Mike C. sits down to chat with Greg Malcom about the Python Koans project, which you need to check out.

So why are you still reading this? Why are you not listening to from python import podcast while playing with Python Koans??

Nothing left to see here..

With the recent chatter about 'DevOps', I thought to myself, how can I explain the idea in a terse manner?  It's pretty simple, really!  You have a business need or problem, so you hire a developer to write code that will solve it.  Web Operations is a business need (or problem, depending on how you approach it), and you hire a sysadmin to do some crazy magic to solve it.

Looks like the presentations from PyOhio 2010 are published over at blip.tv.  That's awesome, now I can catch up on the ones I missed.

Below is my presentation, chopped and hacked to make me look like less of an idiot ;)

Also, you can find my slides here: Splunking With Python and the source code for the talk with examples here: pyohio_talk @ bitbucket

Overall I had a great experience at PyOhio and can't wait for next year!

So, yeah, PyOhio was all kinds of awesome. So awesome, in fact, that it was tool damn cool for 2010. What you say? I mean to say that according to the handout, we were actually in 2011. You had to be there to believe it. Sucka.

This is only the second PyOhio I have been able to attend, although I’ve managed to be involved in the most minuscule way since it’s inception. First of all, I love the venue, The Ohio Union. I found it to be accessible, convenient, and appropriate for our needs. The Round Room seating situation could be improved, but I think that’s already on the agenda for next year.

The first talk I caught on Saturday was one that really piqued my interest. Scott McCarty talked about log analysis and demod a tool named petit, a pretty awesome log analysis tool written in Python. It basically gives some really practical ways to help your logs get you the info you need in less time, with less pain.

To entertain her, I sat in on Sarah’s talk, which is a really nice introduction to Python for .Net developers. Pretty sure that’s at least the second time I’ve seen that talk ;).

The evening after was great. Columbus has a decent night life! The wife and I strolled around the Arena District and the Short North on High St. We had dinner at The Black Olive and then The Burgundy Room for a drink and Tapas.

Also hung out with a crowd of folks from the conference for a few drinks at the Surly Girl.

Conclusion? Both of the districts are very cool, I’d go back any time for a night on the town.

On Sunday I presented my talk. Initially I had planned to speak on how to use the Splunk Python API/SDK. Sadly, that didn’t work out so well. I started out writing my own interface to the RESTful API that they provide/publish. Then, I discovered that Spunk had done all that already. The problem is that the code doesn’t feel Pythonic to me, and the documentation is just aweful (see this).

While I’m on the subject, I’ll take a moment to rant. One of my peeves in software design is that programmers don’t make clean interfaces. There’s nothing overly hard or complicated about talking to a RESTful service, but they have abstracted it in such a way that it takes digging to figure out WTF is going on under the hood. I would have tried to write the code with an interface that was abstracted, but didn’t leave you wondering what kind of magic Unimacorn dust was thrown in to make it all work. Am I wrong to think that way?? No.

Here is what I have to do for a basic search with Splunk

from splunk import auth, search

if __name__ == '__main__':
    # First of all; MOTHER FUCKING CAMEL CASE
    # Lastly, you call getSessionKey BUT it caches
    # the key, so you don't need to call it again.
    # Therefore, this method name feels like a lie.
    auth.getSessionKey('username', 'password')

    # Because the kw argument list is fucking huge.
    args = {'search': 'search sourcetype="syslog"',
            'latest_time': 'rt',
            'earliest_time': 'rt'}

    # Monolithic method to "dispatch" things.
    # I am not a fucking trucker, or police officer.
    search_job = search.dispatch(**args)

    # Pandas are now crying bloody tears
    if search_job.isDone:
        for index, event in enumerate(search_job.events):
            print event

Were I the developer responsible for the Splunk interface, I would have designed it something like this:

from splunk import Auth, Search

if __name__ == '__main__':

    auth = Auth()
    auth.authenticate('username', 'password')
    restful_search = Search()
    search_job = restful_search.basic_search(realtime=True,
                     sourcetype='syslog')

    # I like generators..
    while True:
        try:
            print search_job.events.raw.next()
        except StopIteration:
            break

Anyway, I’m over that now, moving on.

So, my talk went pretty OK. Like I started to say earlier, the Splunk part didn’t pan out. I still touched on the basics of the Splunk API and what I didn’t like. Most of the slides I wrote, however, were before I started seeing things I didn’t like about the code, so people were likely confused.

I also talked briefly about a suite of tools we use internally at AGI for managing ~1k servers. It’s called cf and is loosely based on cfengine, but built in Python with a bias toward flexibility and decoupling from the build process. Parts of the system are open source, but the goal is to open it completely and release a stable version.

In contrast, I also spoke about Fabric to basically show how simple application deployment can be.

After my talk, I stuck around for calvinhp’s talk about Supervisord, one of my favorite Pythonic unix tools. One I’ve even contributed back to!

You can find my slides here: Splunking With Python and my code here: pyohio_talk on bitbucket

So in a nutshell, PyOhio is awesome, and you should go every year.