I’m still looking for lots more goodies. I have yet to find the perfect off-road running stroller, and I’m a bit disappointed by the selection of baby snowboards, but I’ve got a few more months to sort it all out.

Here’s some of the clothes and gear I’ve found so far…

Nerdy T-shirts:

Exothermic reaction t-shirt – clevercuties.com

Bibonacci t-shirt – clevercuties.com. The nerd in me cannot resist something this awesome.

Big boys clothes in little man sizes:

Baby track suit – Nordstrom’s. Oh heck yes, he gonna be a soccer playa!

Baby Chuck’s – zappos.com

Outdoor gear:

Full-face helmet – nybikergear.com. This is a prerequisite for the snowboard, dirt bike, skateboard, and baby sky-diving outfit that he’ll be donning after a few months.

Kid in a bag – rei.com. Before he’s able to walk, he’ll be able to tag along on hikes. Throw him in a bag on my back and off we go. They even sell sun visors and other accessories.

Snowsuit – backcountry.com. He’s gonna be the hippest kid on the slopes for sure. Gotta look good, especially considering he’ll almost certainly be the youngest person to ever bust a double cork.

It provides two methods that you can use in your code, isVisited and isAnyVisited.

One important note about your URL specification is that you must specify an *exact* URL that the user has been to. So, for example, if someone has visited several twitter profiles, but never actually went to the homepage or signin screen, then it would be difficult/impossible to tell whether or not they were a twitter user without a massive brute-force query.

You can download sniff.js, or view a demo.

Sample usage:

dojo.require("mdg.sniff");
var usedYahoo = mdg.sniff.isVisited("http://www.yahoo.com");
var usedGoogleMaps = mdg.sniff.isAnyVisited([
      "http://maps.google.com",
      "http://maps.google.com/maps"]);
var usedFacebook = mdg.sniff.isAnyVisited([
      "http://www.facebook.com",
      "http://www.facebook.com/index.php",
      "https://login.facebook.com/login.php"]);

Source code:

dojo.provide("mdg.sniff");
 
//
// Browser history sniffing, based on infamous blog post:
// <http://www.niallkennedy.com/blog/2008/02/browser-history-sniff.html>
//
// Sample usage:
//
//    dojo.require("mdg.sniff");
//    var usedYahoo = mdg.sniff.isVisited("http://www.yahoo.com");
//    var usedGoogleMaps = mdg.sniff.isAnyVisited([
//          "http://maps.google.com",
//          "http://maps.google.com/maps"]);
//    var usedFacebook = mdg.sniff.isAnyVisited([
//          "http://www.facebook.com",
//          "http://www.facebook.com/index.php",
//          "https://login.facebook.com/login.php"]);
//
// Works with Dojo 1.3 and 1.4 (*may* work with 1.2 as well)
//
 
dojo.require("dojox.html.styles");
 
(function(){
 
    var _this = this;
 
    this.sniffCache = {};
 
    dojox.html.insertCssRule(".dojohistorysniff a", "color:#000000;");
    dojox.html.insertCssRule(".dojohistorysniff a:visited", "color:#ff0000 !important;");
 
    this.isAnyVisited = function(/*Array*/urls) {
        for (i=0; i<urls.length; i++) {
            if (_this.isVisited(urls[i])) {
                return true;
            }
        }
        return false;
    };
 
    this.isVisited = function(/*String*/url) {
        if (typeof(_this.sniffCache[url]) != 'undefined') {
            return _this.sniffCache[url];
        }
        var link = _this.addLink(url);
        var color = new dojo.Color(dojo.style(link, "color"));
        if (color.r == 255) {
            _this.sniffCache[url] = true;
            return true;
        }
        _this.sniffCache[url] = false;
        return false;
    };
 
    this.insertSniffDiv = function() {
        return dojo.create("div", {className: "dojohistorysniff"}, dojo.body());
    };
 
    this.getSniffDiv = function() {
        var divs = dojo.query("div.dojohistorysniff");
        if (divs.length > 0) {
            return divs[0];
        }
        return _this.insertSniffDiv();
    };
 
    this.addLink = function(/*String*/url) {
        var div = _this.getSniffDiv();
        return dojo.create("a", {href: url}, div);
    };
 
    /**
     * mdg.sniff.isVisited
     * Check whether or not a URL has been visited
     * @param url String
     * @return boolean
     */
    mdg.sniff.isVisited = this.isVisited;
 
    /**
     * mdg.sniff.isAnyVisited
     * Check whether or not *any* of the URLs specified have been visited
     * @param urls Array of Strings
     * @return boolean
     */
    mdg.sniff.isAnyVisited = this.isAnyVisited;
 
})();

I’ve cleanup up that sample code and tucked it into a script. You can pass a list of files to the script, and it will apply a sepia effect to each, making sure to backup the originals.

#!/usr/bin/python
"""
Apply sepia filter in batch to images
 
Usage:
    python batch_sepia.py [--no-backup] file1 [file2] ...
"""
 
import Image as PIL_Image
import shutil, os
from optparse import OptionParser
 
 
def open_image(filename):
    """ grab a PIL image from the given location
    """
    image = PIL_Image.open(filename)
    image.load()
    return image
 
def save_image(image, filename, quality=95):
    """ save the PIL image to disk
    """
    image.save(filename, "JPEG", quality=quality)
 
def make_linear_ramp(white):
    """ generate a palette in a format acceptable for `putpalette`, which
        expects [r,g,b,r,g,b,...]
    """
    ramp = []
    r, g, b = white
    for i in range(255):
        ramp.extend((r*i/255, g*i/255, b*i/255))
    return ramp
 
def apply_sepia_filter(image):
    """ Apply a sepia-tone filter to the given PIL Image
        Based on code at: http://effbot.org/zone/pil-sepia.htm
    """
    # make sepia ramp (tweak color as necessary)
    sepia = make_linear_ramp((255, 240, 192))
 
    # convert to grayscale
    orig_mode = image.mode
    if orig_mode != "L":
        image = image.convert("L")
 
    # optional: apply contrast enhancement here, e.g.
    #image = ImageOps.autocontrast(image)
 
    # apply sepia palette
    image.putpalette(sepia)
 
    # convert back to its original mode
    if orig_mode != "L":
        image = image.convert(orig_mode)
 
    return image
 
def convert_image(filename, make_backup):
    """ convert an image at the given path to sepia tone.
        @param filename
        @param make_backup - if True, will copy original file to file.bak
    """
    if not os.path.exists(filename):
        print 'Skipping %s' % filename
        return
    print 'Processing %s...' % filename
    if make_backup:
        shutil.copyfile(filename, '%s.bak' % filename)
    save_image(apply_sepia_filter(open_image(filename)), filename)
    print 'Done.'
 
def convert_images(files, make_backup=True):
    """ convert the list of filenames to sepia tone.
        @param filename
        @param make_backup - if True, will copy original file to file.bak
    """
    map(lambda f: convert_image(f, make_backup), files)
 
 
if __name__ == '__main__':
    parser = OptionParser()
    parser.add_option("-x", "--no-backup", dest="no_backup", default=False,
            action="store_true")
    (options, files) = parser.parse_args()
    convert_images(files, make_backup=not options.no_backup)

You can execute this from the command line as:

$ python batch_sepia.py image1.jpg image2.jpg image3.jpg

If you want to recursively apply the filter to a bunch of images, you might consider mixing this with some find/xargs-fu:

$ find $HOME/pictures -name "*.jpg" | xargs python batch_sepia.py

Once you’ve got your photos in order, head over to photoworks.com to get them printed!</shameless-plug>

#!/bin/bash
if [ -z "$1" ]; then
    echo "usage: $0 revision"
    exit
fi
 
REV=$1
 
# Execute a svn diff on the revision, and search the output for deleted lines
# (begin with a '-', but aren't followed by another immediately),
# and likewise with any added lines
DIFF="$(svn diff -c $REV --diff-cmd=/usr/bin/diff)"
DEL_COUNTS=$(echo -e "$DIFF" | egrep '^[-][^-]' | wc -lm)
ADD_COUNTS=$(echo -e "$DIFF" | egrep '^[+][^+]' | wc -lm)
 
function get_field {
    # given an input string with 2 fields delimited by spaces, possibly with
    # leading whitespace, return the field in the position specified.
    # so,  `get_field "   123   45678" 1`  would return "122"
    # and  `get_field "   123   45678" 2`  would return "45678"
    INPUT=$1
    POSITION=$2
    echo $INPUT | sed "s/^[ ]*\([0-9]*\)[ ]*\([0-9]*\)/\\$POSITION/g"
}
 
DEL_LINES=$(get_field "$DEL_COUNTS" 1)
DEL_CHARS=$(get_field "$DEL_COUNTS" 2)
 
ADD_LINES=$(get_field "$ADD_COUNTS" 1)
ADD_CHARS=$(get_field "$ADD_COUNTS" 2)
 
echo "Removed $DEL_LINES lines ($DEL_CHARS characters)"
echo "Added $ADD_LINES lines ($ADD_CHARS characters)"

Save this as $HOME/bin/svnstat, then execute it passing in a revision, e.g. $ svnstat 1001

$ svnstat 61765
Removed 113 lines (4913 characters)
Added 63 lines (2975 characters)

If you want to then see your daily net lines of code, hook this up to an egrep‘d svn log, piped into xargs.

Update 1/13/2010:
I cleaned up the “get_field” function to use a single sed command with backreferences rather than piping the string through tr, sed, and cut.

Also updated the initial calculation of DIFF_COUNTS and ADD_COUNTS to only require 1 single svn diff execution. Used echo -e to solve the newline issue I was having on a first failed attempt.

Disclaimer
I’ve only spent a grand total of about 3 hours researching and coding anything flash-related. Any information distributed here could lead you down a long and lonesome path — however, more likely, it’ll get your feet wet enough so that you can take off on your own adventure.

Prerequisistes:

1. Download and install the Adobe Flex SDK. I’m using Adobe Flex version 3.4 on Ubuntu 9.10
2. Have basic familiarity with a command line terminal (Linux, cygwin, etc.)

Good so far? Great. Here’s what I hope to help you understand, because it was what I struggled with.

1. Filesystem layout for your application
2. Writing your application, coding mxml and ActionScript
3. Packaging and signing your application
4. Installing and running your new application

There are a lot of other more advanced topics that I’d like to learn myself, including but not limited to:

1. Using third party libraries (.swc’s)
2. Unit testing your application with ASUnit
3. Upgrading your life with ANT buildscripts

Steps to writing your first air application:

1. Create a directory to hold this project and its source code.

       $ mkdir myairapp
       $ mkdir myairapp/src

2. Generate a new .mxml file inside your source code folder.

       $ vim myairapp/src/MyAirApp.mxml

   FlexBuilder admittedly helps a ton here, but you don’t need it. This file defines what your application will look like, using XML markup to indicate panes, widgets, and all sorts of fun stuff. However, there’s no reason it can’t be done in VIM or your favorite text editor. This file (along with your actionscript files) will likely be the most volatile part of the project as you continue to code. Here’s a basic template you can download and use. Stick this in your root source folder.

   <?xml version="1.0" encoding="utf-8"?>
   <mx:WindowedApplication xmlns:mx="http://www.adobe.com/2006/mxml" xmlns:ns1="*"
           layout="absolute" width="300" height="200" showTitleBar="false" showStatusBar="false"
           currentState="load_state" creationComplete="init()">
       <mx:states>
           <mx:State name="load_state"></mx:State>
       </mx:states>
       <mx:Script>
       <![CDATA[
           private function init():void {
           }
       ]]>
       </mx:Script>
   </mx:WindowedApplication>

3. Update the “mx:states”, which will be used to represent different viewports/panes within this app. Each state will have a button, and it will be a different color for each. The first, named “load_state”, has a green button as defined by the “fillColors” attribute. The second, named “clicked_state”, has a blue button.

        <mx:states>
           <mx:State name="load_state">
               <mx:AddChild position="lastChild">
                   <mx:Button x="24" y="36" width="250" height="122" fillAlphas="[1.0, 1.0]"
                               fillColors="[#00FF00, #00FF00]" label="Page 1"/>
               </mx:AddChild>
           </mx:State>
           <mx:State name="clicked_state">
               <mx:AddChild position="lastChild">
                   <mx:Button x="24" y="36" width="250" height="122" fillAlphas="[1.0, 1.0]"
                               fillColors="[#0000FF, #0000FF]" label="Page 2"/>
               </mx:AddChild>
           </mx:state>
       </mx:states>

4. Write some actionscript code to toggle between states when you click the buttons. The currently visible one can be read and changed based on a global variable, `currentState`.

           private function onState1ButtonClick():void {
               // Go to the 2nd "state" when this button is clicked
               currentState = "clicked_state";
           }
    
           private function onState2ButtonClick():void {
               // Return to the initial "state" when this button is clicked
               currentState = "load_state";
           }

   Then, wire-up the events so that clicking will actually trigger the code. Add a “click” attribute to the “mx:Button” element to match

               <mx:Button ... click="onState1ButtonClick()" ... />
               <mx:Button ... click="onState2ButtonClick()" ... />

5. The last thing you’ll need before you can play with your masterpiece is your “application.xml” file. Stick this in the main project directory, rather than in with your source code.

       $ vim myairapp/MyAirApp-app.xml

   See the Adobe AIR documentation for an example.

Packaging, code signing, generating .air, and installing
Your app is *almost* ready to run. The basic steps to get the app transformeed from source to binary are:

1. Generate .swf with amxmlc
2. Generate code signing certificate with adt (alternatively, if you’re a linux whizbang you can use openssl)
3. Tweak your application .xml. Link to .swf from step #1
4. Package it up into .air with adt

I’ve created a shell script to automate the package, code signing, and generating air step. Copy this into “myairapp/build.sh”

#!/bin/bash
APP_NAME=MyAirApp
FLEX_BIN=/opt/flex_sdk_3.4/bin
 
# build a .swf
mkdir release
$FLEX_BIN/amxmlc ./src/$APP_NAME.mxml -output ./release/$APP_NAME.swf
 
# generate code signing certificate
$FLEX_BIN/adt -certificate -cn $APP_NAME 1024-RSA cert.pfx $APP_NAME
 
# package into .air
$FLEX_BIN/adt -package -storetype pkcs12 -keystore cert.pfx \
    -storepass $APP_NAME \
    ./release/$APP_NAME.air ./$APP_NAME-app.xml ./release/$APP_NAME.swf \
    # Uncomment the following lines if you have icons that you have created
    #-C assets icon_16.png \
    #-C assets icon_32.png \
    #-C assets icon_48.png

Now execute the build script. Assuming there were no errors, your new application is ready to be installed. Open up the newly generated .air file in the release directory, and the air installer should walk you through the install process.

If you want to start getting fancier, the next step you’ll want to do is externalize your ActionScript code. All you have to do is create a “.as” file (or an entire directory if you want to break up your code into namespaces), stick it next to the .mxml, and then you’ll be able to “import” it and use functions, classes, etc from it. I prefer to keep the actual event wiring within the .mxml code, but don’t let it get too cluttered.

Additionally, you’ll want to start adding assets (icons, images, sound files, etc) to your project. Create a separate “assets” directory to hold them.

Additional Flex development references

• http://opensource.adobe.com/wiki/display/flexsdk/Flex+SDK
• http://www.adobe.com/devnet/flex/
• http://flex.org/
• To find great examples, search Google for “flex site:googlecode.com”

More help with compiling and packaging:

• http://www.adobe.com/devnet/air/articles/signing_air_applications_print.html
• http://blog.devsandbox.co.uk/?p=163
• http://livedocs.adobe.com/flex/3/html/help.html?content=compilers_13.html#150640

Best of luck!

:inoremap # X<C-H>#

See http://vim.wikia.com/wiki/Restoring_indent_after_typing_hash

Here are some notes I jotted down while at the conference. It’s a bit overwhelming, but I wanted to throw this out there for now, then I’ll come back and reorganize it.

Tutorial: PyCon 401 – Some Advanced Topics, by Steve Holden
* String interpolation
* Iterators
* Generators
* Descriptors and name lookups
* Metaclasses

———————————-
String Interpolation
———————————-

Can use locals() trick

def fmt(date, x):
    amount = 3
    return "%(date)12s $(x)-25s %(amount)8.2f" % locals()

———————————-
Iteration
———————————-

Iterable object must have next() and __iter__()
* next returns next item, or raises StopIteration
* __iter__ returns an “iterable” object (e.g. self)

———————————-
Generators
———————————-

Any function with “yield” in it is automagically assumed to be a generator by
the compiler
* Never actually return anything
* Obey the iterator protocol, and thus have a next() method

def g(s):
  for c in s:
    yield c+'!'
g1 = g('hello')
assert [c for c in g1] == ['h!', 'e!', 'l!', 'l!', 'o!']

———————————-
Decorators and other cool shit
———————————-

functools.wrap
* when making a decorator, decorate your decorator with this to preserve
“dunder_name” __name__ and dunder-doc __doc__

functools.partial
* copy a function to a new place and provide different kwarg defaults

———————————-
Descriptors and Properties
———————————-

HACKATTACK: Property definition without namespace polluion
* see ~/src/sandbox/src/pycon/tests/test_property.py

Descriptor protocol just talks about how to look up stuff in a class
* not all that cool :-/

———————————-
Metaclasses
———————————-

factories
* produce and return functions
* produce and return classes

metaclass
* can do cool shit to a class without cluttering up the class definition
* keep your classes closer to application domain model and responsibilities

Example metaclass: Automagically Decorate Every Method in Class
* page 35 of notes defines a Tracer class factory with __new__, that wraps
every callable, non-dundered (__x__) function with a tracing decorator
* you add a __metaclass__ attribute to your class, then when it gets spun up,
the tracer factory decorates its shit
* see ~/src/sandbox/src/pycon/tests/test_metaclass.py

Notes from Friday

———————
Windmill
———————
in-browser, cross-browser functional testing framework
* started as fork of selenium
* great visual recorder
* dom inspector cross-browser
python api
continuous integration w/ hudson
reporting hooks
issues:
* no ssl support

http://www.getwindmill.com

LOOKS AWESOME!

———————
Profiling
———————
Optimization makes code harder to maintain
But necessary sometimes because Python is slow in real world
Premmature optimization is the root of all evil. Forget about small efficiencies. Focus on hot spots.
Improve your algorithms for the 3% of worst things
* unroll loop
* add caching
Use profiling to figure out which pieces are the 3%

track entry and exit times to functions using Timer

How to:

import cProfile
cProfile.run("python.swallow(gopher)")

? pstats.Stats(‘1.profile’, ‘2.profile’)
** not good for threaded systems… need seperate profiler per thread

KCacheGrind can help explore profile output (runs on wxpython)

Look for
* way too many calls
* large local time in single function (can be spot-optimized)
* large cumulative per call (possibly cache results)
* high waiting times (IO blocked ?)

———————
Testing AJAX Web Apps
———————
It’s hard, and Kumar has failed. This is just an overview of what has/hasn’t worked.

Example: pylons+dojo aintjustsoul.net
* ajax to fetch album thumbnails

Strategy:
– test data handlers (in python)
– test javascript
– lint it first for syntax errors (e.g. javascriptlint.com)
– rhino runner (perhaps with john resig’s env.js to simulate DOM)
– spidermonkey ?
– Selenium, jquery.simulate, YUI.test to simulate user events in browser
(now it gets trickier)
– isolate UI for testing
– DI your connection stuff to mock out data and use a fake server, at the
cost of a little more indirection
– automate UI tests
– selenium remote control automates browser interactions via python code
– gridify your tests
– “selenium grid” to speed up tests
– future: test swarm
– crowd sourced continuous integration
– seems like an awful security flaw

http://bit.ly/testing-ajax (mailing list on google groups)

JSBridge can help to launch firefox from terminal
* it fakes X-server so mozilla runs but doesn’t put anything to display
* you can still run your tests in an actual browser rather than rhino’s implementation
* mozilla ubiquity plugin tested this way

———————
Auomated QA Infrastructure
———————

Ingredients:
* VCS
– mainline branch.. merges trigger full QA battery
– devs use QA system on dev branches
* reporting
– if you don’t do this, nothing else matters
– “your QA system is ONLY as good as its reporting of results”
* tests (discovery, coverage metrics, static source analysis, performance profiling)
– discovery = nose
– coverage = figleaf
– static analysis = pylint
– profiling = ????

CI frameworks:
* svnchecker (svn post-commit hooks)
* bitten
* pybots
* buildbot ***** <-- best general purpose python-based open-source framework now

Buildbot Architecture
* master: coordinate, dispatch, report
* build slaves: runs tests as directed
- BuildFactory defines the build process
>>> factory.addStep(SVN(url))
>>> factory.addStep(Compile(command=['python', 'setup.py', 'build'])
>>> factory.addStep(nose.run()) #????
# they used Trial(testpath=’.') for Twisted
>>> factory.addStep(Figleaf(…
>>> factory.addStep(PyLint(…
* changes: encapsulates events that require testing
* schedulers: makes decisions about when/where to order tests
* pollers/triggers: signals the base change events
– built-in SVNPoller

———————
Twisted/MQ/RPC
———————
… (notes in moleskin) …

———————
Testing large, untested codebases
~ Titus
———————

Example: biological something or other
on the science background of code under test
* example: pygr – python graph database
– provides data abstration with pygr.Data
* problem in biology
– tons and tons of data, parsed lists, etc

on the code and authors
* da code
– ~8k of Py, ~2k of Pyrex
– almost all library and framework, no UI fluff
* the people
– written by 1 dude ==> lots of technical debt
– no tests, interface contracts, etc.
– code ownership is issue by new people that want to contribute.
don’t want to step on original author’s code
– idiosyncratic code since author was a smart dude

statement: code coverage is of limited utility bc it doesn’t measure branch coverage
* TITUS SAYS WRONG WRONG WRONG
* it’s not just bc it doesn’t measure branch coverage. there are other more
valid reasons why it might be flawed

grokking code through coverage
* howto:
– given a big chunk o python
– import it, write test to exercise some of it, check coverage to see what it done
– add additional statement to test
– repeat
* lets you pull apart highly complex/coupled code
*

problem: no good way keep track of tests in a large codebase
* some things may or may not be getting run by your runner
* new project “tattle” that keeps track of master list of tests
– when running, checks the master list and if something is missing or new, it warns you
– not decided how to keep track of what’s changed
– yaml ?
– tag the source ?
– check the diff ? (requires “saving” coverage state, then comparing on next run)

“for those that use nose, each additional dot is like another hit of smack”

code review:
* pep8
* pep257 – docstring format!!!

“each time you’ve pissed on another line of code, it’s like it’s yours now”

reformatting vs “search/replace” vs real functional changes
* what you gonna break?

coverage is an invaluable lever for prying into other people’s code bases

what’s next?
* trace code coverage between commits (and which lines are no longer covered)
* branch coverage (titus on GSoC project: “i have no idea how to do this, that’s what students are for”)
* simplify CI. buildbot too complex

Notes from Saturday

——————————
!!!! LIGHTNING TALKS !!!!
——————————

== thestatusisnotquo.com ==
@mpirnat’s socially redeeming python mini-community of hackers

== sqlpython ==
write sql in unix command shell
have access to all other cool unix stuff (grep, sed, cat, etc)

== webob ==
presentation was cool… just played through the doctests in a
fake interactive interpreter. flawless demo FTW!

== DVCS ==
mercurial (hg) is super easy to use
statement: hg on bitbucket is better than svn on google code
successful DVCS don’t really use the D
but django 1.1 delayed by a month because of DVCS crap
what’s the benefit:
* better merging (but shouldn’t require DVCS)
* easier repository creation (as opposed to svnadmin)
svn isn’t quite dead yet (but it’s not getting better)

== pymite ==
python on microcontrollers.. runs on 8bit devices with 64kb ram
easily embed native code in docstrings

== web2py ==
MVC web framework. used for pycon registration.
online DB admin tool ala django
online python code editor
integrates with bug ticketing system and links back to original stack trace
templating system looks like advanced pyfusion

——————————
Guido’s keynote
——————————
py3.1/2.7 coming
no decision on dvcs
tool to backport from py3 to py2 coming. then we can write code in 3 and give it to the 2x world also.

——————————
State of Django
——————————
… (see moleskin) …

** statement: everyone should be using sphinx for technical docs **

——————————
Lack of Design Patterns
——————————
“python isn’t java without the compile”

define lack of patterns:
* comp.lang.python 100k messages
* visitor/flyweight/strategy/state pattern was most often referenced
(but still less than “sausage”)

why no talk of patterns?
* patterns are built-in
– first class functions
– metaprogramming (can manipulate functions/definitions)
– iterators
– closures (even if not a “true” closure in python, but close enough)

examples:
* factory
>>> {‘thing1′:Thing1, ‘thing2′:Thing2}[which_one]()

conclusions:
* look to features of langauge before patterns
* reduce patterns –> shortercode
* if need a pattern, add a language feature (e.g. concurrency stuff)

presentor wants “channel pattern” (as a model for processes to replace threads)
* look at PyCSP (or stackless?) for concurrency

——————————
Class Decorators
——————————

just like function decorators
* definition: a callable that accepts at least one argument and returns something

available as of py2.6

def identity(ob):
    return ob 
 
@identiy
class C():
    pass

in 2.5, you can do same type of thing, but you can’t use the @decorator syntax

class decorators are not inheritable!!!
but you can stack them on top of a single class
(as opposed to metaclasses, which are inherited but not stackable)

default method decorator vs. mixins (eg partial class)
* ???

popular patterns (besides just replacing what was done with metaclasses)
* register
* augment (mixin replacement)
– example: take the __lt__ method and automagically make an __eq__ and
__gt__ for class
* fixup (monkeypatching decorator)
– example: replace all dunder-attribute with single underscores, so can get
access to the stuff that was privatized. can monkeypatch a decorator into
a thirdparty library too! (named @stop_doing_java)
* verify (assert stuff about the class)

registration pattern
* presentors best use case
* add stuff to a list that’s kept elsewhere

@cron.schedule(cron.NIGHTLY)
class SalesReport(Report):
    def run(self):
        #blah

——————————
ORM Panel
——————————
…

——————————
Scripting in Python (super beginner)
——————————

Compromise Layout pattern
Project
/bin
script.py
/scriptlib
__init__.py
scriptimpl.py
setup.py
Conditional main pattern
>>> if __name__ == ‘main’:
… sys.exit(main(sys.argv))
3 layers of IO pattern:
* should you deal with filenames, file objects, or generators ? YES!
* main should take a filename. invokes something that takes a file instanece.
below that use generator layer for efficiency.
Use optparse pattern
wtf. not exactly what i was expecting from this talk. time to go drink.

——————————
Drop ACID and think about data
——————————
ACID: a promise ring your DBMS wears
* atomicity – all or nothing
* consistency – no explosions during crashes/etc
* isolation – no fighting from writers, can pretend transactions are serial
* durability – no lying to you.. assume things worked

scalability and reliability: requires a bunch of stuff from rdbms to get there

CAP theorem says pick 2:
* consistency, availability, partition tolerance

BASE:
* basically available, soft state (doesn’t have to be consistent immediately), eventually consistent
* Google BigTable
– append-only data makes it faster to insert
– fantastic data compression due to hybrid row/column storage
* Amazon Dynamo key/value store
– consistent hashing to distribute data to appropriate node
* Facebook Cassandra
– OPEN SOURCE!! incrementally scalable
– no compression, not quite polished

distributed db’s are the new hotness, but none are awesome yet.
* we don’t need another half-baked dynamo clone

memcached: key/value store used as cache
* RAM only. throws data away. lightning fast. everyone’s doing it.
* invalidation is hard and failure-prone
– be careful with mutable data. consistency will be lost over DB solution.

Tokyo Cabinet/Tyrant: super enhanced berkeleydb key/value store
* disk persistent, but still very performant
* actively developed
* replication similar to mysql
!! possible solution for photoworks session

document databases
* schema free, easy to use,
* easier to migrate. some documents will just have different properties.
– but application still needs to be able to deal with this
!! we kind of got this with photoworks project meta-data
* couchdb – document db poster child
* mongodb – way faster than couchdb. json/bson (binary json-ish)

column databases:
* sequential reads are awesome
* compresses much better than rows
* monetdb – DON’T USE THIS

——————————
Concurrency and Distributed systems
——————————
concurrent > parallel > distributed

goals:
* decrease latency, increase throughput

issue?
* global interpreter lock (GIL). one thread at a time.
– ugh. slow. boo.
– perhaps fixed in unladen-swallow ?
* but you can build concurrent regardless of GIL

multiprocessing in 2.6+
* processes and IPC via pipes to allow parallelism
* similar API to thread/queue

more on GIL workarounds…
* Jython
– allows free-threading via java.util.concurrent, at the cost of losing C extensions
* IronPy
– also unrestricted threading, and some C extensions via ironclad
* PyPy
– complete rethink of interpreter
– not production ready

Twisted
* asynchronous, event driven multitasking
* supports thread usage, but may not always be threadsafe

Kamaelia
* easy to understand. very easy to get up and running. IPCs, Threads, etc abstracted away
* compontents talk vis mailbox. cooperative multitasking via generators
* lots of fantastic examples
!!! try this out

problem w/ frameworks
* twisted/kamaelia radically different
* you end up adapting the methodology in app, which can be dangerous

messaging/rpc flooded with libraries
* Pyro for RPC should be checked out
* memcached for shared memory
* Stomp, ApacheActiveMQ for message queues
– space flooded b/c heavy use by web apps for asynch processesing right now

Notes from Sunday

—————————————–
Paver
—————————————–
build process helpers
* cmd line
* config
* work w/ files
* svn, sphinx, virtualenv, etc.
* distutils, setuptools

without paver, you end up writing a seperate script for everything you want to do

$ easy_install Paver
$ paver paverdocs #displays Paver documentation

build files…
$ vi pavement.py

options(
    setup = setup_meta,  #via setup.py
    virtualenv=Bunch(
        packages_to_install=["", "", ""],
...

tasks are the heart of paver

@task
def clean():
   """ cleans up paver dict,  removes virtualenv tree and build dir """

other fun decorators too
@cmdopts – integration w/ optparser to set up cmd line options for your tasks

paver.easy
* logging, unix command running, dry-run stuff against command line

paver.path
* jason orendorff’s path.py module built in

paver.doctools
* sphinx (generate html docs)
* ned batch’s Cog support. “section off” pieces of code and pull into your docs dynamically.

paver.ssh, paver.svn, paver.virtual (virtualenv)

deployment of code delegated to pytoss (?)

getting started: http://bit.ly/starting_paver

—————————————–
Functional Testing Panel
—————————————–
windmill
* python proxy – avoids same domain security in browser
* python browser launching
* python CI reporting
* javascript simulate user interactions – tries to get it right (bubbling, mouseclick events)
* javascript waits for dynamic content
* javascript assertions

selenium:
* “record/playback is the root of all evil”. none of creators wanted to do it
* next version will have more native C hooks. javascript event faking could only get so far.
* really only good at testing in firefox, rest of browsers (including chrome) suck

twill:
* technically just a small DSL on top of mechanize/clientform/beautifulsoup
* forms, cookies, redirects, link checking
* but no javascript support. ever. not even a little.

webtest (ian bicking)
* runs fake request/responses on WSGI frameworks
* written to test paste
* doesn’t actually run http

from myapp import wsgi_app
from webtest import TestApp
app=TestApp(wsgi_app)
app.go('/')
print app.resp

nose
* nose.cfg for default cmd line opts
* –with-django plugin to help set up env

py.test 1.0
* distributed testing. central reporting.
– run your tests on all different envs (clouds, o/s’s, py versions, etc)
* plugins similar to nosetests. but not documented all that well
* runs unittests, doctest, restdoc, pocoo (sends failures to pastebin)
* tetamap.wordpress.com, pytest.org

vision: make automated deployment merge w/ actual deployment, and have success
be mandated for deploy.

—————————————–
Lightning talks
—————————————–

== argparse ==
all features of optparse, but better docstrings and never touch sys.argv
sub command parsers (e.g. svn add) easier than optparse
mutually exclusive arguments

http://code.google.com/p/argparse

== python 3.1 ==
some performance improvements. IO now almost as fast as 2x
unittest improvements. assertRaises trick
OrderedDict finally arrives

from __future__ import bad_syntax_ideas
* braces, no colons, no indentation, implied ruby blocks, etc.
* pep 3117

== Poor Man’s Continuous Integration ==
try to do it without buildbot using your existing tools
* capastrano, maestro, git/svn
using “fabric” instead of “git push”
$ fab push -d staging_server deploy
* wraps up the test run and git push steps

== cluemapper – software project manager ==
example: lots of software projects
* each w/ issue tracker, svn
* want to share users across projects
built on top of trac. “trac for teams”
* basic reporting and time-tracking
* trac plugins to extend core
* wsgi + dojo ajax
components
* irc bot
* theming
* pypi egg server
* bzr integration
* pastebin
* timetracking, reporting
nice source control user manager for each repository
roadmap
* agile/scrum/xp stories and whiteboard
– drag/drop between stories and iterations
!!!!!!!!!!!!!!!!!!!!!!! yayz !!!!!!!!!!!!!!!!!!
cluemapper.org

== ReleaseBot ==
release mgr process watches buildbot
push button to tag a version for deploy
* workflow for notifying QA that this is what needs tested
buildbot notices tag and preps for release
not yet developed, just idea phase

== pyjamas ==
python to javascript compiler, with built-in widget set
JSONProxy seems fun!
rest is pure evil.

== \ is evil ==
don’t split lines with a backslash
use parentheses instead

my_str = ("asdlfkjasdf   asdlfkjdsf"
        "  asdflkj   asldkjf "
        " alskjdf asdlkjj ")

insetad of

evil_nasty_str = "asdlfkjasdf   asdlfkjdsf" \
        "  asdflkj   asldkjf " \
        " alskjdf asdlkjj "

Mini-conclusion
Things for follow-up:
* windmill. first round of experimenting at openspace didn’t prove to be as awesome as i had hoped. but i’m willing to give it another try. scripting these in python feels like an afterthought, and it wouldn’t be easy to get integrated with BuildBot.
* ClueMapper. looking forward to the scrum/xp tools built into Trac.
* webtest instead of twill for scripting end-to-end tests
* py.test for distributed testing
* tokyo cabinet for key/value store with persistence
* web2py web framework. the data schema admin manager doesn’t feel quite as polished as django, and i don’t like the online editor. but their MVC ideas and templating implementation might be useful.
* mercurial or bzr. i need more practice with DVCS. i committed all my code throughout the weekend to a local mercurial instance, but never pushed up to a central repository.

That’s all she wrote for now. Looking forward to future PyCon’s and to trying out some of these awesome tools.

I don’t think it’s any shock that Google has the data to segment users based on their browsing habits. The shocker to me is that they’re making the result of that segmentation available to webmasters everywhere.

The full text of the email is:

From: Google AdSense
Subject: Introducing interest-based advertising – action required for your AdSense account

Hi,

We’re writing to let you know about the upcoming launch of interest-based advertising, which will require you to review and make any necessary changes to your site’s privacy policies. You’ll also see some new options on your Account Settings page.

Interest-based advertising will allow advertisers to show ads based on a user’s previous interactions with them, such as visits to advertiser website and also to reach users based on their interests (e.g. “sports enthusiast”). To develop interest categories, we will recognize the types of web pages users visit throughout the Google content network. As an example, if they visit a number of sports pages, we will add them to the “sports enthusiast” interest category. To learn more about your associated account settings, please visit the AdSense Help Center at http://www.google.com/adsense/support/bin/topic.py?topic=20310.

As a result of this announcement, your privacy policy will now need to reflect the use of interest-based advertising. Please review the information at https://www.google.com/adsense/support/bin/answer.py?answer=100557 to ensure that your site’s privacy policies are up-to-date, and make any necessary changes by April 8, 2009. Because publisher sites and laws vary across countries, we’re unfortunately unable to suggest specific privacy policy language.

For more information about interest-based advertising, you can also visit the Inside AdSense Blog at http://adsense.blogspot.com/2009/03/driving-monetization-with-ads-that.html.

We appreciate your participation and look forward to this upcoming enhancement.

Sincerely,

The Google AdSense Team

Email preferences: You have received this mandatory email service announcement to update you about important changes to your AdSense product or account.

Google Inc.
1600 Amphitheatre Parkway
Mountain View, CA 94043

One thing that has me thinking they’re less than purely evil is that they at least show you what they think you are in your personal ad preferences page, and allow you some options for configuration. However, it would take someone reasonably savvy to locate that page.

I’m interested to see where this goes. I know it’s a big win for businesses that want better ad targeting, but this is clearly a big setback for privacy advocates.

Regardless, I really can’t wait to see the ad profile they assign me… . o O ( Computer nerd, mountain biking, porn addicted, binge drinker )

OMBC Race Series

1. Mohican Wilderness – Loudonville (March 29)
2. Vulture’s Knob – Wooster (April 18)
3. Mount Wood Challenge – WV (May 3)
4. The Wilds – Zanesville (May 17)
5. Eastfork State Park – Bethel (June 28)
6. S&S Trails – Mt. Perry (July 12)
7. Alum Creek State Park – Delaware (July 26)
8. Lake Hope SP – Nelsonville (August 2)
9. Caesar Creek SP – Waynesville (August 23)
10. Dillon XC Race – Zanesville (August 30)
11. Westbranch SP – Ravenna (September 27)
12. Mohican State Park – Loudonville (October 11)
13. Mohican Wilderness – Loudonville (October 25)

Endurance Events:

1. Mohican MTB100 (May 30)
2. Mohican Trail 100 (June 20-21)
3. 12/24 Hours of Mohican (September 19-20)

* Dates are subject to change. See www.ombc.net for the latest and greatest info.

After fighting with Ubuntu Hardy 8.04 for months to get WIFI working, I finally found a Dell driver [0] that worked on my HP laptop through ndiswrapper [1].

A few hours later with success claimed, I decided to upgrade to Ubuntu Intrepid 8.10, assuming I had conquered all possible obstacles and that anything in the future would be pithy. Little did I realize, I was in for a world of hurt, as Ubuntu decided to muck with the guts of video/display drivers between 8.04 and 8.10.

After an initial upgrade, I was stuck with 800×600 resolution using the crappy built-in display drivers. I tried to activate the restricted Nvidia driver, but it hung and nothing happened. I went to Nvidia’s website, downloaded their open-source driver, installed it as they suggested, and rebooted. It looked great at first — the resolution was back to normal at the login window, but as soon as I logged-in, the screen went blank. Stumped, I pressed ctrl+alt+f1, then ctrl+alt+del to reboot. I went into the terminal, restored the original /etc/X11/xorg.conf.backup, and was able to get back in, but back to 800×600.

Two full days later, I’ve finally got 1280×800 resolution back on my laptop. It appears that the auto-detected settings from Nvidia in my xorg.conf put in faulty horizontal sync and vertical refresh settings.

The proper settings are:

    HorizSync       30.0 - 110.0
    VertRefresh     50.0 - 150.0

With that, I was able to enable the nvidia driver and boot into gnome successfully.

Next, I decided to re-enable some desktop effects through the appearance manager. I opened it up, enabled the medium-level of effects, and immediately got the f$@#%%ng black screen!!! BLASTED!!! A reboot got me back to where I started.

This whole process is so frustrating. Luckily, xkcd can provide some comic relief…

So that’s where I’m at now. Hopefully I’ll figure something out soon, and update this post for any other HP laptop owners to reference.

Reference:
[0] Dell wifi driver that worked with the HP Pavilion dv2715nr’s 14e4:4315 Broadcom card ftp://ftp.us.dell.com/network/R174291.exe
[1] Instructions for ndiswrapper can be found at https://help.ubuntu.com/community/WifiDocs/Driver/Ndiswrapper