Wednesday, August 11, 2010

Sikuli at Ohio LinuxFest

And now, for my next trick, one month from today, I shall demonstrate how to script virtually any program with a GUI... without touching an API! (audience gasps)

If You Can See It, You Can Automate It: Sikuli
Your computer should save you from doing tedious and repetitive tasks yourself, but automation programming has been too difficult for casual uses. MIT's Project Sikuli changes that. It fuses traditional text-base scripts with actual screenshots of targets on your screen, making programming simple, versatile, and useful. If you've ever told a friend, "Go here, now click on this, then on that", you know enough to start using Sikuli. Let Project Sikuli shake up your notions of what programming is like.

I'll be spending most of this month using Sikuli to build a test suite for a complex GUI app at work, recording the pain points as I go. I'd also appreciate your questions and comments about Sikuli - I may be able to work them into my talk (or at least be prepared when they're asked from the audience.)

Ohio Linux Fest, Sep. 10 - 12.

5 comments:

Bruce said...

Sounds good, my idea is you can demo the cross platform ability of Sikuli, this should impress most your audience.

David said...

"I'll be spending most of this month using Sikuli to build a test suite for a complex GUI app at work, recording the pain points as I go"

Just wanted to say that there are probably many of us who appreciate you (and everyone) who spend the time to examine new concepts and/or give talks on Python. We all benefit from that.

shi said...

Sikuli is a an awesome idea, but still
some questions come to mind (nothing you wouldn't have thought of yourself though):

how well does it work with
- dynamic UIs
- localized UIs
- themed/skinned UIs

How hard is it in practice to make a script robust enough ? (e.g. detecting unexpected error windows, taking into account that with a non-maximized window certain controls can be invisible, etc etc) ?

How useful is it to write GUI tests? (Can I get the contents of an edit control?)

Catherine said...

Thanks for the comments so far, everybody!

shi, some answers:
1. Great - if (and only if) the controls still look the same (or fairly similar - the visual matching obeys a tuneable quality-of-match parameter.)
2. ditto
3. Big fat FAIL, unless you do some serious bulletproofing, like writing your script to reset the skin at the beginning. (You can also use text controls to work around some of the issues.)
4. All perfectly do-able, but you'll
need real programmer skills and real programmer persistence to do it - the "easy" part doesn't apply to bulletproofing.
5. It's great for GUI tests, though reading text off the screen requires a bit of a hack: http://sikuli.org/blog/2010/05/06/extract-text-with-clipboard/

shi said...

The clipboard trick is quite cool!
Thanks for pointing me to that blog entry.