Monday, September 13, 2010

pdb with Sikuli

As I mentioned in my last post, the big barrier to debugging a Sikuli script with pdb is that, when you enter a debugging command like "next", "continue", etc. that is supposed to allow program flow to continue, you are in the debugging console window, and your Sikuli script will send the keyclicks and mouse actions into the debugging console window instead of the application your script was supposed to be driving.

Here's a minor modification to pdb that can change that by using Alt-TAB to switch back to the application window every time program flow moves from the debugger to the application. Save this code as in the Lib directory that you unzipped Sikuli into, for example, ~/Sikuli-IDE/Lib. (/Lib is not created by default, but it is the first item in sys.path.)

#! /usr/bin/env python

"""A Python debugger.

A slight modification of ``pdb``; attempts to switch
screen focus back to the invoking app before running commands
in the program being debugged. This should prevent Sikuli
commands from being "typed" or "clicked" into the debugger
window when they should go into the window of the app being
# (See pdb.doc for documentation.)

import pdb
import sys

class Pdb(pdb.Pdb):
def _to_running_window(self):
"""Moves focus to the window a Sikuli script was running
before the debugger was invoked, so that subsequent commands
will act in the app instead of the debugger."""
self.onecmd("SCREEN.type(Key.TAB, KEY_ALT)")

def do_step(self, arg):
return pdb.Pdb.do_step(self, arg)
do_s = do_step

def do_next(self, arg):
return pdb.Pdb.do_next(self, arg)
do_n = do_next

def do_return(self, arg):
return pdb.Pdb.do_return(self, arg)
do_r = do_return

def do_continue(self, arg):
return pdb.Pdb.do_continue(self, arg)
do_c = do_cont = do_continue

def set_trace():

Now, at the point in your Sikuli script where you want to enter the debugger, insert
import sikpdb; sikpdb.set_trace()

Also remember that you need to add -Dsikuli.console=false to your .sh or .bat script in order to allow pdb input/output to appear in the console window you used to invoke Sikuli. Otherwise, Sikuli will just hang when you attempt to enter the debugger (it's waiting for your input - in some nonexistant interactive session). For instance:
$ cat
DIR=`dirname $0`
java -Xms64M -Dsikuli.console=false -Xmx512M -Dfile.encoding=UTF-8 -jar $DIR/sikuli-ide.jar $*

The remaining problem is that depends on the notion that ALT-TAB will reliably return you to the application's window. This is true, but only if you do not browse around among your open windows while in the debugger; if you do, you'll change the z-ordering of the windows. It would be more reliable to call switchApp() with the title of the application's window, but I don't know of a cross-platform way to automatically detect it - the best I can think of would be to manually pass it as an argument to set_trace, have set_trace store it in an instance variable like self._application_window_title, then call self.onecmd("switchApp('%s')" % self._application_window_title) in place of self.onecmd("SCREEN.type(Key.TAB, KEY_ALT)") . It's more complex to use, but more reliable... feel free to use that approach if you prefer it.

Sunday, September 12, 2010

Sikuli talk follow-up

Thanks to everybody who made time in a busy Ohio LinuxFest schedule to make it to my talk on Project Sikuli! I appreciated your attention and your questions.

You can browse my slides at, and/or download a tarball with all the code demonstrated at

I promised to follow up on the questions that stumped me, so here are the first couple answers...

- I mentioned how debugging with pdb is difficult because switching to the command-line window to interact with pdb interrupts the program flow, and when you issue a "continue" in the debugger, your Sikuli script will barrel along in the command-line window with the debugger instead of in the application you wanted Sikuli to drive. A couple people suggested debugging the script from a remote machine, connecting with pdb and attaching the debug process to the Sikuli process. I still don't know whether that's possible - definitely maybe - but I realized that there's a very simple remedy that I feel silly for not suggesting before.

In your Sikuli script at the point you want to enter the interactive debugger, instead of simply inserting import pdb; pdb.set_trace(), insert import pdb; pdb.set_trace(); switchApp('Window Title Of The App I Am Debugging') - that should put the script you're debugging back on track.

EDIT: WAIT - I got this wrong - this doesn't make sense. You do need to switchApp() back to the window your script was running in, but not immediately after invoking pdb.set_trace(); rather, it needs to be invoked just before running, pdb.continue, etc. I think the right way to do this is to subclass pdb; give me a bit to produce and test something like that.

- I confirmed that there isn't yet a "Recording" mode, or anything to automatically translate your actions into Sikuli commands. Really, though, I'm not sure how much that would help - writing the Sikuli commands is actually the easy part of the automation. The hard part is figuring out precisely which actions to take to get the effects you want most reliably.

OLF was really good this year. I especially enjoyed the hallway track - met lots of really interesting people that I need to follow through with on some neat ideas. The exhibit hall was a great place for PyOhio folks to meet new people and connect them with the Python community - thanks so much for all the PyOhio crew who came out to staff the booth! Thanks to the organizers for all their work!

Thursday, September 09, 2010

Quran troll

Ian Bicking hit it right on the nose: this Quran-burning pastor is a troll, no more.

Arguably, I shouldn't even be posting this. Trolls should be ignored. The trouble is, even internet-savvy people have trouble living up to that prescription. The world as a whole definitely doesn't know what to do about trolls. There are already people in Afghanistan holding demonstrations because of this troll... so I feel like "just ignore it" maybe isn't enough right now.
John Shimek: The best suggestion I have heard it to ignore it and donate towards the Pakistan flood relief.
Very good. But I'm wondering if one step further would be a good idea: an explicit linkage. You know, "I pledge a $x donation for Pakistan flood relief for every copy of the Quran burned." Isn't there a webservice out there that facilitates making this sort of pledge? Help me out, lazyweb - this should happen fast if it's going to happen.

Granted, Pakistan flood relief shouldn't be an issue of the victims' religion one way or the other, so there's something a little odd about using it as an anti-bigotry statement. And maybe this does amount to feeding the troll. But at least it would feed some real-live human beings at the same time. It's the only way I can think of to turn this stupidity into something good.

And, if you can permit me just one little troll-foodish comment...

To be an a**hole is one thing. (I really struggled to find another word here. This time, though, it really is the only word.) But to say, "It's not me - it's God who's the a**hole, and I'm just obeying him"... I don't understand how anybody can say that without being 100% flat-out certain that there is no God. Why would anybody take the slightest chance of ever standing before God and having Him say, "Could you repeat that to My face, please?"