Some of you may have noticed that BitSyncHub (and my company homepage) went down for almost three days this week. Now, I’m proud of my skills building highly available, fault-tolerant, state-of-the art and bleeding edge systems, deployed on OS, vendor and geographically redundant infrastructure, that never goes down. Sometimes a failover may take a short while, or parts of a system may get bogged down and queues build up, but the system, while on it’s knees, may never fail.
Since I got several requests for BitSyncHub to support BitBucket Git repository synching to GitHub, I went ahead and added the functionality. The service will dedtect the appropriate repository type, and push specified branches - although the source branch will be ignored for now, so a branch speficiation of ‘foo:bar’ will simply push ‘bar’. To make this happen, I finally had to bring gitapi a bit closer to completion, so I released the first version to PyPi for general consumption as well.
Introducing BitSyncHub Since I’m an automation nut, when I found Travis CI, I was understandably excited - automatic running of my testcases for hgapi from the repository as opposed to a pre-push hook (as I have had it set up since the beginning of time) would avoid the oh-so embarrassing mistakes of forgetting to add a new file to the repository and having a non-working version in the repo. I just have to set up some service to synch to the GitHub mirror and all will.
I’ll be in Florence for EuroPython 2013 and do (more or less) a follow-up to the training session I held last year - a very hands-on venture into Python lanugage and standard library features that will allow you to implement your bad ideas in awesome hacks and good ideas with beautiful magic. This is how fun we had last year! Ok, so they’re all looking at their screens, but that’s kind of the point with a training session in my opinion, hands on keyboards as much as possible.
The other day I needed to download some zip files, unpack them, parse the CSV files in them, and return the data as dicts. I did the very same thing a couple of years ago, and although the source is lost, I recall having a Python (2.4?) script of about two screens to do the download - so a hundred lines. When re-implementing the solution now that I know Python and the standard library better, I ended up with 12 lines written in just a few minutes - edited for blogging clarity it clocks in at 13 lines:
When we started with fleet management at Visual Units, one thing was really hard to get right - distance calculations. There was no end of information available, but most-to-all of it was on a level of mathematics far beyond a poor developer who feels that anything beyond discrete mathematics and basic geometry and statistics really should be somebody else’s problem. The implementations that could be found were closed-source licensed version we really could not afford at that stage.
(part 1, part 2, part 3, part 4) I’m almost done with the parts to make this project PyPI ready - it can now work on your application as long as you implement the actual code to route calls to the right part of your API - this works: python -m pyrest.launcher –server pyrest.integration.cherry \ –api=pyrest.examples.hg Configuration can also be done using a config file instead of or together with command-line arguments.
(part 1, part 2, part 3) I’ve now split the code into separate parts - pyrest.py now only has generic functionality for hooking and routing, along with a bunch of helpers to create responses with HTTP response types. In fact, it’s only 35 lines of code, and that’s the entire ‘core’ of pyRest so far. The CherryPy integration has moved to the pyrest.integration package as cherry.py - it’s still pretty clumsy to use (python -m pyrest.
In part 2, I hooked up the API to CherryPy in a very crude fashion, and this time we’ll look at how we can add handlers for resources in a less clumsy way. I decided to keep handlers on one ‘level’ only - that is, /sketch/parrot and sketch will both be handled by the /sketch handler. This is because I find that the same sub-resource often is present in several places (what about //props/parrot?
In part 1, a very unexciting base CherryPy implementation was all we had, but now it’s time to hook up something real! Instead of creating a mock API to work against as example code, I’ve decided to use hgapi to access the pyrest repo itself as example implementation - very meta! I’ve decided to hook the API in before I refactor the code to separate the web framework from pyRest, since I firmly belive in getting things working first and cleaning up after.