Benjamin W. Smith // Sysadmin by trade, Pythonista by passion. Dad to two boys and a girl. Guitarist. I like my coffee black, just like my metal.
Looks like the presentations from PyOhio 2010 are published over at blip.tv. That's awesome, now I can catch up on the ones I missed.
Below is my presentation, chopped and hacked to make me look like less of an idiot ;)
Also, you can find my slides here: Splunking With Python and the source code for the talk with examples here: pyohio_talk @ bitbucket
Overall I had a great experience at PyOhio and can't wait for next year!
So, yeah, PyOhio was all kinds of awesome. So awesome, in fact, that it was tool damn cool for 2010. What you say? I mean to say that according to the handout, we were actually in 2011. You had to be there to believe it. Sucka.
This is only the second PyOhio I have been able to attend, although I’ve managed to be involved in the most minuscule way since it’s inception. First of all, I love the venue, The Ohio Union. I found it to be accessible, convenient, and appropriate for our needs. The Round Room seating situation could be improved, but I think that’s already on the agenda for next year.
The first talk I caught on Saturday was one that really piqued my interest. Scott McCarty talked about log analysis and demod a tool named petit, a pretty awesome log analysis tool written in Python. It basically gives some really practical ways to help your logs get you the info you need in less time, with less pain.
To entertain her, I sat in on Sarah’s talk, which is a really nice introduction to Python for .Net developers. Pretty sure that’s at least the second time I’ve seen that talk ;).
The evening after was great. Columbus has a decent night life! The wife and I strolled around the Arena District and the Short North on High St. We had dinner at The Black Olive and then The Burgundy Room for a drink and Tapas.
Also hung out with a crowd of folks from the conference for a few drinks at the Surly Girl.
Conclusion? Both of the districts are very cool, I’d go back any time for a night on the town.
On Sunday I presented my talk. Initially I had planned to speak on how to use the Splunk Python API/SDK. Sadly, that didn’t work out so well. I started out writing my own interface to the RESTful API that they provide/publish. Then, I discovered that Spunk had done all that already. The problem is that the code doesn’t feel Pythonic to me, and the documentation is just aweful (see this).
While I’m on the subject, I’ll take a moment to rant. One of my peeves in software design is that programmers don’t make clean interfaces. There’s nothing overly hard or complicated about talking to a RESTful service, but they have abstracted it in such a way that it takes digging to figure out WTF is going on under the hood. I would have tried to write the code with an interface that was abstracted, but didn’t leave you wondering what kind of magic Unimacorn dust was thrown in to make it all work. Am I wrong to think that way?? No.
Here is what I have to do for a basic search with Splunk
from splunk import auth, search if __name__ == '__main__': # First of all; MOTHER FUCKING CAMEL CASE # Lastly, you call getSessionKey BUT it caches # the key, so you don't need to call it again. # Therefore, this method name feels like a lie. auth.getSessionKey('username', 'password') # Because the kw argument list is fucking huge. args = {'search': 'search sourcetype="syslog"', 'latest_time': 'rt', 'earliest_time': 'rt'} # Monolithic method to "dispatch" things. # I am not a fucking trucker, or police officer. search_job = search.dispatch(**args) # Pandas are now crying bloody tears if search_job.isDone: for index, event in enumerate(search_job.events): print event
Were I the developer responsible for the Splunk interface, I would have designed it something like this:
from splunk import Auth, Search if __name__ == '__main__': auth = Auth() auth.authenticate('username', 'password') restful_search = Search() search_job = restful_search.basic_search(realtime=True, sourcetype='syslog') # I like generators.. while True: try: print search_job.events.raw.next() except StopIteration: break
Anyway, I’m over that now, moving on.
So, my talk went pretty OK. Like I started to say earlier, the Splunk part didn’t pan out. I still touched on the basics of the Splunk API and what I didn’t like. Most of the slides I wrote, however, were before I started seeing things I didn’t like about the code, so people were likely confused.
I also talked briefly about a suite of tools we use internally at AGI for managing ~1k servers. It’s called cf and is loosely based on cfengine, but built in Python with a bias toward flexibility and decoupling from the build process. Parts of the system are open source, but the goal is to open it completely and release a stable version.
In contrast, I also spoke about Fabric to basically show how simple application deployment can be.
After my talk, I stuck around for calvinhp’s talk about Supervisord, one of my favorite Pythonic unix tools. One I’ve even contributed back to!
You can find my slides here: Splunking With Python and my code here: pyohio_talk on bitbucket
So in a nutshell, PyOhio is awesome, and you should go every year.
