I was reading Kill Zone by Gunnery Sgt. Jack Coughlin, and was struck by the personal mantra of one of the Marine snipers:

Slow is smooth, smooth is fast.

Even though lives are not usually on the line, it’s important in the software industry to understand that sometimes we shouldn’t rush around like chickens with our heads cut off. During your project’s crunch time, this is most important.

Slow down, take a breather, and think before you slam in a bunch of crap code. All the technical debt you create has to be accounted for by someone. Be the rabbit, not the hare.

We used to build software like a crappy banana split stand. Grab the banana off the shelf (which probably is about to expire), throw on ice cream, cover it in chocolate, add some cherries, whip cream, nuts, stir it up, put a spoon in it, hand it to the customer, take their money, and let them walk away.

STOP THAT RIGHT NOW!

Stop the gargantuan effort up front, stop compromising the quality of pieces that don’t get seen, stop making your customer wait til the end of the project.

Find a beautiful banana and show it to your customer. Let them take a bite. That might be all they need. Or they might want to take a small piece of it and try it in banana bread. They might not like that. But that’s OK, because we have lots of time and lots of banana left. Take the next piece and make a banana smoothie for them. Maybe add a strawberry for the next sip.

Aaron Oliver might suggest that a good time to have these tastings is right after your daily stand-up.

Let your customer be involved at each step. Let them help you make the right changes early and often. Be agile. Don’t be the hobo behind the bad banana split counter.

That is to say: do one thing, do it well.

Decorators (not to be confused with the decorator pattern) can add behaviors or side-effects to a method, and this can be dangerous. It seems harmless, because by adding a decorator, you’re likely taking boilerplate code and shuffling it elsewhere. However, they often encourage badness because of how easy it is to add these behaviors.

On testing decorated methods:

Adding an @decorator to a python object unarguably makes the undecorated code difficult to test in isolation. Decorators are applied at compile-time, and cannot be mocked or made to not-execute without some pain.

There are certainly a few common tricks that can help test a decorated method with minimal side-effects, but they require changes to the decorator itself. There’s just plain and simple no way to un-decorate a method for isolated testing.

Let’s look at a few common examples:

@expose: register a URL for a view function in a web framework

class BlogPostController(object):
    @expose("/blog/{post_id}")
    def index(self, request, post_id=None):
        """ show a blog post """
        post = adapter.get_post(post_id)
        return render("show_entry.html", dict(post=post))

Without the `@expose`, your `show_entry` knows how to get a given post and render it in the proper template. With the decorator, it also now knows which URL corresponds to that. You now have multiple reasons for this block of code to change, including pointing to a different URL or using a different template. It’s preferred to have a separate module for managing which urls point to which views.

Harm factor: low. Ick factor: medium – high.

@cache: try to get the result from memcache, otherwise, execute the function and stick it in cache for next time

class PostAdapter(DataAdapter):
    @cache
    def get_post(self, post_id)
        """ grab a blog post from the database """
        return self.query(Post).filter(id=post_id).one() # SQLAlchemy folks need to talk to Mr Demeter...

OK this seems cool right? You only hit the database when you have to, otherwise we get it even quicker by looking it up in the cache.

What happens if you want to disable caching? A separate cache abstraction layer would reduce volatility in your data adapter.

And what does the unit test look like?

class TestGettingAPost(object):
    def setup(self):
        self.query = Mock() #don't hit the production database!
        self.post_adapter = PostAdapter(query=self.query)
    def test_getting_a_post(self):
        assert self.post_adapter.get_post(123)

Damn, there’s no way to mock out the @cache decorator so it doesn’t run. Try to, I dare you. You’re likely going to actually get post 123 from your production memcache. Crappy. The only thing you can do is make the @cache grab the cache implementation from the PostAdapter instance (and mock that out in you test), or find some other sneaky way of disabling caching for test runs. But the @cache decorator isn’t all self-contained and fun anymore.

Harm factor: medium – high.

@validate: Make sure the request matches the specified schema, otherwise hand-off to error handler

class EditPostController(object):
    def _save_error(self, request, errors):
        """ @validate decorator kicked flow here, redisplay edit page with errors """
        ...
    @validate(schema, error_handler=_save_error)
    def save(self, request)
        post = self.schema.to_python(request.params)
        self.post_adapter.save_post(post)
        return redirect("/blog/{0}".format(post_id))

Without this @validate, there would be a lot of boilerplate code inside the `save` method. With it, any unit test for the save method will be likely linked to your schema. You’d have to make an actual valid request in order to test this method. That’s outside of any tests for your schema directly. That means double-coverage but 2 tests to update when requirements shift.

Harm factor: low-medium

Conclusions (or tl;dr):

As easy as it is to become infatuated with Python decorators, they definitely encourage you to violate the SRP. This can create a myriad of problems:

• Difficultly in isolating system under test
• Added complexity to enable testing
• Redundant redundant unit tests
• Making a code module more volatile than it ought to

They still have some valid use cases and can lead to cleaner code, however, as Master Yoda once said, “when you look at the dark side, careful you must be…”

My favorite driver is an outspoken dutch guy. He’s quick on the wheel and if he doesn’t get the idea I’m trying to convey to him he’ll just type something to try if that works. When he does get what I’m mumbling it’s on the screen faster than when I would have typed it myself, so it doesn’t give me time to get frustrated over things not being like the would be when I had the keyboard. And that gives me time to look for exits and pittfalls.

He also dismisses anyone that tries to say pairing isn’t as effective as coding solo.

No you’re not faster on your own, you’re just creating more crap for your colleagues to puzzle over and eventually delete. The code you write alone sucks. That guy that is getting on your nerves is trying to tell you (clumsily) that your code sucks, try to listen to him and you’ll turn into a better programmer. Or maybe you can teach him something and he’ll stop getting on your nerves. … If you’re slowing the other guy down, that’s a good thing. That will prevent him from writing code that you cannot maintain. If you don’t feel worthy of your colleagues code, get over it, or get off the team.

My biggest stumbling block to date has been the ratio of time driving to navigating when you’re pairing a senior and junior dev. Trying to balance productivity and mentoring isn’t always easy. I find allowing the Sr. dev to drive 2/3 of the time and Jr. dev to drive 1/3 strikes close to the optimal balance. Any more driving for the Sr and you’ll lose the Jr entirely as he rides along for the tour. Any less and you lose valuable time in guiding-by-doing.

Whether you’re the driver or navigator, take your role seriously. Share in the responsibility and help foster a mutual respect for the other.

Chad had high hopes early. He bought the truck on a whim and, without too much inspection, figured it’d be a few month job to get it on the road again. Needless to say, months passed, and it was still in his dad’s garage. Life (and college) got in the way, and the truck sat abandoned for several years, with only an occasional wrench gracing it’s steel.

He’d buy new parts from time to time, and store them away to be put on at a later date. He’d spend the occasional weekend tearing something off, grinding on it, patching, and welding. Tearing off one part would show deeper damage. This was Chad’s baby, and he couldn’t simply mask that damage, so he’d plan to fix the next issue before continuing, or order another new body panel. Little visible progress was made, but lots of hours and dollars were invested.

Finally, 3 weeks ago, after lots of pressure from his friends and family over the last few years, Chad took the truck out of storage at his parents 2 hours away and brought it to his own garage. He’s got a reinvigorated passion to get this thing running, and I’m excited to help him.

Some of these challenges sound familiar to software veterans, but are any analogies actually relevant?

Consider for a moment that you’re assigned to a project that requires you to migrate an existing web application from Java+Oracle to PHP+MySQL. Doing rewrites of applications can often be driven by a desire to consolidate platforms and reduce costs, and there is often a fixed timeline. The project is locked and loaded, and you haven’t had time to do technical due diligence.

You dive right in without much planning, and before you know it, you’ve passed your deadline. You got hung up digging into the guts of the old software. What little new code you were able to write relies on features that your new database doesn’t have, but you didn’t realize because you were testing against the old one.

What has agile software development taught us that can right this project-gone-bad?

It would be a stretch to suggest the core tenets of the agile manifesto are entirely applicable. There really aren’t any processes/tools getting in the way, he’s not held up by a desire to document everything about the truck, contract negotiation doesn’t matter, and he really doesn’t have much of a plan that would inhibit responding to change.

However, the principles behind an agile team can be very useful to this situation.

• Rapid, continuous delivery of a useful software truck.  Don’t worry if, for example, the interior dome lights aren’t working right away.  But let’s get it running, and make sure it’s poised to keep getting better.
• A working software truck is the principal measure of progress.  Not how many new body parts got ordered off ebay, or how cool the sketch of flames on the doors might look.
• The project truck is built by motivated individuals, who should be trusted.  The friends that help need to be committed to the goal.
• Continuous attention to technical excellence and good design.  Don’t rush it.  And avoid duct tape.
• Simplicity.  One of the beautiful things about old cars is how few moving parts there really are under the hood.  Don’t ruin it by adding flamethrowers to the exhaust or an automatic transmission.
• Regular adaptation to changing circumstances.  Expect that we’ll uncover a few unanticipated problems, but be ready to figure them out and keep charging forward.

Chad is a perfectionist, and he’s not in a rush to get this thing done. However, we have made some progress already in the last few weeks. Here’s where we are now: most barriers to working on the truck have been removed, we’ve started a “spike” of work to get the motor close to starting, I’m going to help put together a prioritized list of features (similar to user stories), and we’ll identify incremental milestones along the path to winning car shows.

With any luck, we’ll get the motor fired up after our first sprint. And maybe, just maybe, I’ll learn something along the way that can help us develop better software!

Photo courtesy of chicanerii on flickr

Eh. Doesn’t really do it for me as a blob of color.

How about on a link? Well, that is actually quite pleasant against a white background.

Regardless, the takeaway from the whole story isn’t that you should switch all your colors to #0044cc, but rather that you need to rigorously test your audience and be able to make changes based on the observed behavior. Having the discipline to carefully react to the measurements can and will drive success.

It provides two methods that you can use in your code, isVisited and isAnyVisited.

One important note about your URL specification is that you must specify an *exact* URL that the user has been to. So, for example, if someone has visited several twitter profiles, but never actually went to the homepage or signin screen, then it would be difficult/impossible to tell whether or not they were a twitter user without a massive brute-force query.

You can download sniff.js, or view a demo.

Sample usage:

dojo.require("mdg.sniff");
var usedYahoo = mdg.sniff.isVisited("http://www.yahoo.com");
var usedGoogleMaps = mdg.sniff.isAnyVisited([
      "http://maps.google.com",
      "http://maps.google.com/maps"]);
var usedFacebook = mdg.sniff.isAnyVisited([
      "http://www.facebook.com",
      "http://www.facebook.com/index.php",
      "https://login.facebook.com/login.php"]);

Source code:

dojo.provide("mdg.sniff");
 
//
// Browser history sniffing, based on infamous blog post:
// <http://www.niallkennedy.com/blog/2008/02/browser-history-sniff.html>
//
// Sample usage:
//
//    dojo.require("mdg.sniff");
//    var usedYahoo = mdg.sniff.isVisited("http://www.yahoo.com");
//    var usedGoogleMaps = mdg.sniff.isAnyVisited([
//          "http://maps.google.com",
//          "http://maps.google.com/maps"]);
//    var usedFacebook = mdg.sniff.isAnyVisited([
//          "http://www.facebook.com",
//          "http://www.facebook.com/index.php",
//          "https://login.facebook.com/login.php"]);
//
// Works with Dojo 1.3 and 1.4 (*may* work with 1.2 as well)
//
 
dojo.require("dojox.html.styles");
 
(function(){
 
    var _this = this;
 
    this.sniffCache = {};
 
    dojox.html.insertCssRule(".dojohistorysniff a", "color:#000000;");
    dojox.html.insertCssRule(".dojohistorysniff a:visited", "color:#ff0000 !important;");
 
    this.isAnyVisited = function(/*Array*/urls) {
        for (i=0; i<urls.length; i++) {
            if (_this.isVisited(urls[i])) {
                return true;
            }
        }
        return false;
    };
 
    this.isVisited = function(/*String*/url) {
        if (typeof(_this.sniffCache[url]) != 'undefined') {
            return _this.sniffCache[url];
        }
        var link = _this.addLink(url);
        var color = new dojo.Color(dojo.style(link, "color"));
        if (color.r == 255) {
            _this.sniffCache[url] = true;
            return true;
        }
        _this.sniffCache[url] = false;
        return false;
    };
 
    this.insertSniffDiv = function() {
        return dojo.create("div", {className: "dojohistorysniff"}, dojo.body());
    };
 
    this.getSniffDiv = function() {
        var divs = dojo.query("div.dojohistorysniff");
        if (divs.length > 0) {
            return divs[0];
        }
        return _this.insertSniffDiv();
    };
 
    this.addLink = function(/*String*/url) {
        var div = _this.getSniffDiv();
        return dojo.create("a", {href: url}, div);
    };
 
    /**
     * mdg.sniff.isVisited
     * Check whether or not a URL has been visited
     * @param url String
     * @return boolean
     */
    mdg.sniff.isVisited = this.isVisited;
 
    /**
     * mdg.sniff.isAnyVisited
     * Check whether or not *any* of the URLs specified have been visited
     * @param urls Array of Strings
     * @return boolean
     */
    mdg.sniff.isAnyVisited = this.isAnyVisited;
 
})();

I’ve cleanup up that sample code and tucked it into a script. You can pass a list of files to the script, and it will apply a sepia effect to each, making sure to backup the originals.

#!/usr/bin/python
"""
Apply sepia filter in batch to images
 
Usage:
    python batch_sepia.py [--no-backup] file1 [file2] ...
"""
 
import Image as PIL_Image
import shutil, os
from optparse import OptionParser
 
 
def open_image(filename):
    """ grab a PIL image from the given location
    """
    image = PIL_Image.open(filename)
    image.load()
    return image
 
def save_image(image, filename, quality=95):
    """ save the PIL image to disk
    """
    image.save(filename, "JPEG", quality=quality)
 
def make_linear_ramp(white):
    """ generate a palette in a format acceptable for `putpalette`, which
        expects [r,g,b,r,g,b,...]
    """
    ramp = []
    r, g, b = white
    for i in range(255):
        ramp.extend((r*i/255, g*i/255, b*i/255))
    return ramp
 
def apply_sepia_filter(image):
    """ Apply a sepia-tone filter to the given PIL Image
        Based on code at: http://effbot.org/zone/pil-sepia.htm
    """
    # make sepia ramp (tweak color as necessary)
    sepia = make_linear_ramp((255, 240, 192))
 
    # convert to grayscale
    orig_mode = image.mode
    if orig_mode != "L":
        image = image.convert("L")
 
    # optional: apply contrast enhancement here, e.g.
    #image = ImageOps.autocontrast(image)
 
    # apply sepia palette
    image.putpalette(sepia)
 
    # convert back to its original mode
    if orig_mode != "L":
        image = image.convert(orig_mode)
 
    return image
 
def convert_image(filename, make_backup):
    """ convert an image at the given path to sepia tone.
        @param filename
        @param make_backup - if True, will copy original file to file.bak
    """
    if not os.path.exists(filename):
        print 'Skipping %s' % filename
        return
    print 'Processing %s...' % filename
    if make_backup:
        shutil.copyfile(filename, '%s.bak' % filename)
    save_image(apply_sepia_filter(open_image(filename)), filename)
    print 'Done.'
 
def convert_images(files, make_backup=True):
    """ convert the list of filenames to sepia tone.
        @param filename
        @param make_backup - if True, will copy original file to file.bak
    """
    map(lambda f: convert_image(f, make_backup), files)
 
 
if __name__ == '__main__':
    parser = OptionParser()
    parser.add_option("-x", "--no-backup", dest="no_backup", default=False,
            action="store_true")
    (options, files) = parser.parse_args()
    convert_images(files, make_backup=not options.no_backup)

You can execute this from the command line as:

$ python batch_sepia.py image1.jpg image2.jpg image3.jpg

If you want to recursively apply the filter to a bunch of images, you might consider mixing this with some find/xargs-fu:

$ find $HOME/pictures -name "*.jpg" | xargs python batch_sepia.py

Once you’ve got your photos in order, head over to photoworks.com to get them printed!</shameless-plug>

#!/bin/bash
if [ -z "$1" ]; then
    echo "usage: $0 revision"
    exit
fi
 
REV=$1
 
# Execute a svn diff on the revision, and search the output for deleted lines
# (begin with a '-', but aren't followed by another immediately),
# and likewise with any added lines
DIFF="$(svn diff -c $REV --diff-cmd=/usr/bin/diff)"
DEL_COUNTS=$(echo -e "$DIFF" | egrep '^[-][^-]' | wc -lm)
ADD_COUNTS=$(echo -e "$DIFF" | egrep '^[+][^+]' | wc -lm)
 
function get_field {
    # given an input string with 2 fields delimited by spaces, possibly with
    # leading whitespace, return the field in the position specified.
    # so,  `get_field "   123   45678" 1`  would return "122"
    # and  `get_field "   123   45678" 2`  would return "45678"
    INPUT=$1
    POSITION=$2
    echo $INPUT | sed "s/^[ ]*\([0-9]*\)[ ]*\([0-9]*\)/\\$POSITION/g"
}
 
DEL_LINES=$(get_field "$DEL_COUNTS" 1)
DEL_CHARS=$(get_field "$DEL_COUNTS" 2)
 
ADD_LINES=$(get_field "$ADD_COUNTS" 1)
ADD_CHARS=$(get_field "$ADD_COUNTS" 2)
 
echo "Removed $DEL_LINES lines ($DEL_CHARS characters)"
echo "Added $ADD_LINES lines ($ADD_CHARS characters)"

Save this as $HOME/bin/svnstat, then execute it passing in a revision, e.g. $ svnstat 1001

$ svnstat 61765
Removed 113 lines (4913 characters)
Added 63 lines (2975 characters)

If you want to then see your daily net lines of code, hook this up to an egrep‘d svn log, piped into xargs.

Update 1/13/2010:
I cleaned up the “get_field” function to use a single sed command with backreferences rather than piping the string through tr, sed, and cut.

Also updated the initial calculation of DIFF_COUNTS and ADD_COUNTS to only require 1 single svn diff execution. Used echo -e to solve the newline issue I was having on a first failed attempt.

Disclaimer
I’ve only spent a grand total of about 3 hours researching and coding anything flash-related. Any information distributed here could lead you down a long and lonesome path — however, more likely, it’ll get your feet wet enough so that you can take off on your own adventure.

Prerequisistes:

1. Download and install the Adobe Flex SDK. I’m using Adobe Flex version 3.4 on Ubuntu 9.10
2. Have basic familiarity with a command line terminal (Linux, cygwin, etc.)

Good so far? Great. Here’s what I hope to help you understand, because it was what I struggled with.

1. Filesystem layout for your application
2. Writing your application, coding mxml and ActionScript
3. Packaging and signing your application
4. Installing and running your new application

There are a lot of other more advanced topics that I’d like to learn myself, including but not limited to:

1. Using third party libraries (.swc’s)
2. Unit testing your application with ASUnit
3. Upgrading your life with ANT buildscripts

Steps to writing your first air application:

1. Create a directory to hold this project and its source code.

       $ mkdir myairapp
       $ mkdir myairapp/src

2. Generate a new .mxml file inside your source code folder.

       $ vim myairapp/src/MyAirApp.mxml

   FlexBuilder admittedly helps a ton here, but you don’t need it. This file defines what your application will look like, using XML markup to indicate panes, widgets, and all sorts of fun stuff. However, there’s no reason it can’t be done in VIM or your favorite text editor. This file (along with your actionscript files) will likely be the most volatile part of the project as you continue to code. Here’s a basic template you can download and use. Stick this in your root source folder.

   <?xml version="1.0" encoding="utf-8"?>
   <mx:WindowedApplication xmlns:mx="http://www.adobe.com/2006/mxml" xmlns:ns1="*"
           layout="absolute" width="300" height="200" showTitleBar="false" showStatusBar="false"
           currentState="load_state" creationComplete="init()">
       <mx:states>
           <mx:State name="load_state"></mx:State>
       </mx:states>
       <mx:Script>
       <![CDATA[
           private function init():void {
           }
       ]]>
       </mx:Script>
   </mx:WindowedApplication>

3. Update the “mx:states”, which will be used to represent different viewports/panes within this app. Each state will have a button, and it will be a different color for each. The first, named “load_state”, has a green button as defined by the “fillColors” attribute. The second, named “clicked_state”, has a blue button.

        <mx:states>
           <mx:State name="load_state">
               <mx:AddChild position="lastChild">
                   <mx:Button x="24" y="36" width="250" height="122" fillAlphas="[1.0, 1.0]"
                               fillColors="[#00FF00, #00FF00]" label="Page 1"/>
               </mx:AddChild>
           </mx:State>
           <mx:State name="clicked_state">
               <mx:AddChild position="lastChild">
                   <mx:Button x="24" y="36" width="250" height="122" fillAlphas="[1.0, 1.0]"
                               fillColors="[#0000FF, #0000FF]" label="Page 2"/>
               </mx:AddChild>
           </mx:state>
       </mx:states>

4. Write some actionscript code to toggle between states when you click the buttons. The currently visible one can be read and changed based on a global variable, `currentState`.

           private function onState1ButtonClick():void {
               // Go to the 2nd "state" when this button is clicked
               currentState = "clicked_state";
           }
    
           private function onState2ButtonClick():void {
               // Return to the initial "state" when this button is clicked
               currentState = "load_state";
           }

   Then, wire-up the events so that clicking will actually trigger the code. Add a “click” attribute to the “mx:Button” element to match

               <mx:Button ... click="onState1ButtonClick()" ... />
               <mx:Button ... click="onState2ButtonClick()" ... />

5. The last thing you’ll need before you can play with your masterpiece is your “application.xml” file. Stick this in the main project directory, rather than in with your source code.

       $ vim myairapp/MyAirApp-app.xml

   See the Adobe AIR documentation for an example.

Packaging, code signing, generating .air, and installing
Your app is *almost* ready to run. The basic steps to get the app transformeed from source to binary are:

1. Generate .swf with amxmlc
2. Generate code signing certificate with adt (alternatively, if you’re a linux whizbang you can use openssl)
3. Tweak your application .xml. Link to .swf from step #1
4. Package it up into .air with adt

I’ve created a shell script to automate the package, code signing, and generating air step. Copy this into “myairapp/build.sh”

#!/bin/bash
APP_NAME=MyAirApp
FLEX_BIN=/opt/flex_sdk_3.4/bin
 
# build a .swf
mkdir release
$FLEX_BIN/amxmlc ./src/$APP_NAME.mxml -output ./release/$APP_NAME.swf
 
# generate code signing certificate
$FLEX_BIN/adt -certificate -cn $APP_NAME 1024-RSA cert.pfx $APP_NAME
 
# package into .air
$FLEX_BIN/adt -package -storetype pkcs12 -keystore cert.pfx \
    -storepass $APP_NAME \
    ./release/$APP_NAME.air ./$APP_NAME-app.xml ./release/$APP_NAME.swf \
    # Uncomment the following lines if you have icons that you have created
    #-C assets icon_16.png \
    #-C assets icon_32.png \
    #-C assets icon_48.png

Now execute the build script. Assuming there were no errors, your new application is ready to be installed. Open up the newly generated .air file in the release directory, and the air installer should walk you through the install process.

If you want to start getting fancier, the next step you’ll want to do is externalize your ActionScript code. All you have to do is create a “.as” file (or an entire directory if you want to break up your code into namespaces), stick it next to the .mxml, and then you’ll be able to “import” it and use functions, classes, etc from it. I prefer to keep the actual event wiring within the .mxml code, but don’t let it get too cluttered.

Additionally, you’ll want to start adding assets (icons, images, sound files, etc) to your project. Create a separate “assets” directory to hold them.

Additional Flex development references

• http://opensource.adobe.com/wiki/display/flexsdk/Flex+SDK
• http://www.adobe.com/devnet/flex/
• http://flex.org/
• To find great examples, search Google for “flex site:googlecode.com”

More help with compiling and packaging:

• http://www.adobe.com/devnet/air/articles/signing_air_applications_print.html
• http://blog.devsandbox.co.uk/?p=163
• http://livedocs.adobe.com/flex/3/html/help.html?content=compilers_13.html#150640

Best of luck!