Thursday, May 15, 2008

cmd2 0.3.0

I'm speaking on SQL*Plus alternatives again on Wednesday at GCOUG, and trying to polish up sqlpython a bit in preparation for that. That required polishing cmd2; I've been increasingly embarrassed that cmd2 used homemade code (flagReader) to interpret --options --like --these, instead of using a perfectly good module from the standard library (optparse).

Optparse is not designed for iterative cmd-type apps where you remain at an application prompt after issuing each command - it keeps issuing sys.exit()s to kick you back to the shell. That was my excuse for not using it earlier, but once I decided to fix it, a little subclassing took care of it.

Next I got a little deeper into decorators than I'd ever been, to provide a concise way to specify those options:

@options([make_option('-k', '--ketchup', action="store_true", help="Ketchup on that?"),
[make_option('-r', '--relish', help="relish (sweet or dill)")])
def do_hotdog(self, arg, opts):
if opts.relish:
self.stdout.write('One dog with %s relish\n' % opts.relish)
if opts.ketchup:
self.stdout.write('Hope you're not wearing white!\n')

That was pretty fun. Took some experimentation and this article, though, to figure out how to write a decorator that would accept an argument.

I didn't throw away the homemade ("flagReader") code, for backward compatibility; I issue a DeprecationWarning instead. I did throw away the guts and reimplement it as a wrapper around optparse. I managed to do that part on a single 20-minute bus ride, thanks to good doctests on flagReader. I feel so grown-up!

Finally, inspired by Brandon Rhodes' article in the March 2008 Python magazine, I tried supporting buildout deployments this time. I'm not sure I'm getting it quite right yet; expect it to actually work in cmd2 0.3.1.

Tuesday, May 06, 2008

PyOhio Call for Proposals

PyOhio: Call for Presentations

PyOhio, the first annual Python programming mini-conference for Ohio and surrounding areas will take place Saturday, July 26, in Columbus, Ohio. The conference is free of change and will include scheduled presentations, Lighting Talks and unconference-style Open Spaces.

You can read more about the conference at http://pyohio.org

PyOhio invites all interested people to present scheduled talks. All presentations are expected to last 40 minutes with a 10 minute question-and-answer period. PyOhio will accept abstracts covering any area of Python programming. A classroom area with computers will also be available for possible hands-on sessions.

All presentations proposals should submit abstracts no longer than 500 words in length. Abstracts must include the title, summary of the presentation, the expertise level targeted, and a brief description of the area of Python programming it relates to.

All proposals should be emailed to for review.

The submission deadline will be June 1, 2008. Accepted proposals will be notified by July 1.

Friday, May 02, 2008

choose the tool last

Yesterday, someone asked me if I could design "an MS Access database" for him.

Growl.

As I reflected on it, I realized that the problem is not really the particular technology specified - the problem is that the technology was specified at all before the need was described.

If I offered to make you anything you wanted for dinner, would you ask for "something made with a Teflon spatula?" If you did, I'd tell you, "No no no no no. Describe what you want to eat, and let me figure out what I'm going to use to cook it."

My potential client didn't literally mean that he wanted MS Access to be used, he meant that he wants a database with a GUI interface. He thinks of "Access database" as a way to describe that, because it's the only thing he's ever seen used to build products vaguely resembling the one he's imagining. It's nothing more than a limitation of vocabulary and experience.

The problem is that, so often, that kind of pidgin is taken literally. Tools are pre-selected far too early - while the business users are imagining their needs, before the geeks themselves are even involved,

As a result, there are a lot of geeks out there trying to beat eggs with Teflon spatulas.

My story ends happily, however. My potential client got it when I explained this, and we're going to meet so I can find out what he wants. Then - and only then - will we start discussing which tools to use.

Thursday, April 24, 2008

Penguicon talk materials

Ooh - sorry, I got distracted! Here, at last, are the materials for my talk last Saturday, "Into the Dayta-b'hase: A First-Level Introductory Adventure".

A tarball is here with the slides (dnd.html), queries run (scripts.sql), and a big SQL script to set up the database (dnd.sql)

Monday, April 14, 2008

tax rant

ABSTRACT

Citibank spends a lot of time imploring their customers to go to all-electronic statements... but they don't improve their electronic statement availability to make them practical. Ohio Dept. of Taxation spends a lot of time imploring people to file electronically... but they can't be bothered to test their forms, or to provide a feedback button so that end users can notify them of errors they find.

This is a bad pattern. If you want people to go electronic, focus on getting it right, not on pro-electronic cheerleading.

* * * * *


Ohio has a "use tax" - a law requiring that you add up all your mail-order and interstate purchases from out of state so you can retroactively pay Ohio sales tax on them with your tax return. I think six or seven of us actually obey this law, and the folks at the bureau of taxation laugh, blow a kazoo, and scrawl "PATSY!" on the return whenever they come across someone who paid it. Oh, well.

Digging up records on all these purchases sucks. Citibank, though I love their customer support*, didn't help. (You can download your last six statements, one at a time... or, after March, you can request a PDF report of the whole last year - please allow two business days for your request to be processed. Excuse me? Some people store information on devices called "computers"? A year of credit card statements is about 50 KB... I think you can afford to keep that on disk. Ask Google if you're hard up, they'll give you a few gigs.)

So anyway... I finally, FINALLY, splice electronic records and sift paper records and total up my internet purchases for the year... open up the Adobe form from tax.ohio.gov... and...

...and the line to enter your quantity of out-of-state purchases is not editable. It is pre-set to zero and locked.

One good thing has come out of tonight: I found wesabe, a really nice site for slurping up records from your various accounts automatically. Next year I won't need to fight Citibank, because my Wesabe account will keep itself updated all year long. Wesabe, I'm happy to say, gets it right. Free services generally do, as opposed to people I pay, who generally get it wrong. It's a strange era we live in.

(* - When my credit card was stolen years ago, Citibank was absolutely wonderful to me. It'll take a lot of blundering to erode the goodwill they earned from me then.)

Tuesday, April 08, 2008

PyOhio

It's official! We have a place, we have a date... now all we need is you!

PyOhio
Regional Python programming miniconference
July 26th, 2008
Columbus Metropolitan Library
96 S. Grant Ave.
Columbus, OH 43215 USA

Friday, April 04, 2008

Alas, Mendelson's

Mendelson's is planning to shrink their store and relocate. This is very sad news. I can only hope they don't shrink too much or move too far. Fortunately, they have no timetable for this move yet, and if we're lucky a giant asteroid will hit the earth and wipe us all out first.

If you don't know Mendelson's, the best way I can describe it is "a Jawa sandcrawler parked in downtown Dayton". It's a giant industrial salvage shop, the most amazing pile of thingies and whatsits you can imagine. I consider it the most entertaining location in our fair city. If you can walk through Mendelson's without saying, "MY robot will have THIS!", then you are already dead.

Tuesday, April 01, 2008

Beautiful Visual Python examples

The best thing that can happen, when you try to teach something, is to have somebody learn it and do something that amazes you. That's exactly what happened after my demo of Visual Python at CodeMash... Doug Mair started to play with vPython and came up with two really beautiful examples: a (working!) Rubik's Cube and a set of magnetized disks.

Thank you, Doug!

Monday, March 24, 2008

PyOhio

PyOhio

a regional Python miniconference


Ohio had a good representation at PyCon, and we agreed that the time has come.

http://pyohio.org

announcement-only list: http://mail.python.org/mailman/listinfo/pyohio

planning list: http://mail.python.org/mailman/listinfo/pyohio-organizers

Almost everything is up in the air right now... all we know for certain is that we're aiming for one day, presumably a Saturday, in the Columbus, OH area. Join us, and help the plan take shape!

Friday, March 21, 2008

Teach Me Twisted

I got home from PyCon last night, having stayed for sprints for the first time (I'll blog about that later). It was amazing.

One of the amazing things was Steve Holden's Teach Me Twisted session. I came out of simple curiosity; I witnessed the invention of a whole new form of teaching, I think.

Steve had the front of the room and a laptop with a projector, just as if he were teaching a conventional class - only he wasn't teaching, he was learning. All he had, to begin with, were some ideas of what Twisted could do, not how to do them. "So, how would I use Twisted to check if my website is up?", he began. The Twisted experts sprinkled throughout the audience talked him through it. Steve asked questions to move the session along, and channeled the discussion to keep it focused on his needs as a learner. In particular, when the Twisted experts started waxing eloquent on subtleties or engaging each other in esoteric discussions, Steve cut them off and redirected them back into the task at hand - the task of directing a beginner.

It was wonderfully effective! When you think about it, the standard classroom arrangement gives all the power to the teacher, which is precisely where it doesn't belong, because the teacher is distanced from the learners' needs. By the time you're an expert, you've forgotten what it was like to be ignorant. Bad teachers even forget how to pity the ignorant, using the podium to demonstrate their mastery of the subject to an imaginary audience of fellow masters. Good teachers try to be responsive to the learners' needs, but the arrangement conspires against them. They're like a blindfolded chauffeur, trying to find the road by the verbal instructions of their passengers, who may be too shy to even speak up. It's very hard for a learner to muster the courage to tell a teacher, "I still don't understand what you're saying, and your tangents aren't helping."

By taking the role of the ultimate, empowered learner, Steve turned all that on its head, and made for a session that really was all about the learners' needs to learn. Yet even the experts said afterward that they, too, had learned from it. Perhaps that's because it harnessed the multi-directional, easygoing, collaborative spirit of a good Open Space session, while remaining technical and specific, not wandering into vague chitchat. Bravo, Steve!

Pycon-organizers is buzzing about it, with perhaps overambitious dreams of immediately start a whole series of "Teach Me..." sessions. I'm thinking about trying it out at a local Dynamic Languages SIG meeting. Teach Me Ruby, SIG guys?

Wednesday, March 19, 2008

PyCon Cleveland 2010: Python Rocks!

PyCon is coming to Cleveland in 2010!

OK, I understand there are some other places bidding to host PyCon - some podunk towns like San Francisco and Atlanta - and I'm sure the PyCon organizers will hear their bids attentively, because they're generous people. But, c'mon, you don't seriously think anybody but Cleveland could win, do you? After all, Cleveland is Cleveland, and the others are not!

Anyway, if you want to help bring PyCon to Cleveland, it's time to join the ClePy mailing list and start figuring out how to make it happen.

Saturday, March 15, 2008

Live from PyCon

Yesterday, I was wondering whether this year's PyCon was really turning out to be as amazing as its attendance suggested.

I'm not wondering today. I saw some excellent conventional talks earlier in the day. Mike Bayer showed yummy new SQLAlchemy with infectuous enthusiasm. Kumar McMillan gave a blessedly clear explanation of Unicode in Python. Brandon Rhodes laid out ideas like Adapter patterns very nicely, and showed magnificent stage presence, too.

During discussion of Boston's PyCon 2010 bid:

Steve Holden: "I'm not surprised food is expensive. You send them perfectly good tea, and they throw it in the harbor."

And then, as the pre-scheduled talks ended, the Open Space board went white-hot. I learned a lot at well-attended Open Spaces for DevChix, DBSprockets, and usergroup leaders, and could have enjoyed a dozen other sessions if only I could fork myself.

Right now, I'm finishing my day at Steve's "Teach Me Twisted" session, and wow. I didn't even really care about Twisted, I just wanted to see how the learner-driven idea would work out, and it's going great. About a dozen fat brains and fifty hungry brains are here, and Steve is doing a masterful job of herding us along with skill and humor. How he's managing to absorb this all at the same time as he guides it along with well-placed questions, I don't know. I guess that's what years of teaching experience can do.

Itamar Shtull-Trauring: "Can't we refactor the code a bit more?"
Alex Martelli: "No."

Monday, March 10, 2008

KDE 4: wait

I'm a habitual early adopter. For instance, I moved my production databases to Oracle 11g weeks ago. So, naturally, I had to try the new Kubuntu with KDE 4.0. Sometimes, however, "alpha" really means "alpha", as in "nope, it's not ready yet, leave it alone unless you really want to suffer for science".

So far, these are the features from earlier Kubuntu / KDE combinations that aren't working for me on the new Hardy / KDE4 alpha-alpha.
  • response to volume / mute buttons on hardware
  • Automatic mounting of USB drives (the device doesn't show up under /media until after you've navigated into it with Dolphin)
  • Alt-Tab switching between windows (AAAAAARGH!)
This last one brings it to the threshhold of unusability, and I'll probably uninstall, unless I can find a keyboard shortcut to switching windows soon (and I've had little luck finding a good users' introduction to KDE4). Now, I'm sure there's lots of wonderful stuff about KDE4. I think there's an eternal tension between brilliant designers who improve products and ungrateful users who do nothing but gripe. That's because, if you introduce fifty new features that will please me greatly - when I get around to learning to use them - while, at the same time, breaking five old features that I know and have come to rely on... well, I say the new version "sucks". It isn't fair, is it? But it's the way it works.

Sunday, March 09, 2008

learner-driven Open Space

Over one thousand registrations are in the system for PyCon 2008. Over 2**10, actually. Amazing!

After exulting a bit over that kind of growth, I moved on to worrying about whether PyCon can keep the magic that makes it so great as it grows. A look through the still-forming Open Space schedule quickly reassured me. For instance, Steve Holden has scheduled an Open Space session called "Teach Me Twisted" - not because he knows Twisted, but because he wants to learn it. So he's challenged the Twisted community to come and teach it, and it looks like they're responding.

Learner-driven content - I love it!

Thursday, February 28, 2008

Introducing Mixed

As far as I know, the Cheese Shop didn't have any way to handle fractions and mixed numbers... until now. Introducing mixed.
>>> print Mixed('-1 1/2') + 2
1/2
>>> print Mixed('3/4') + Mixed('1/2')
1 1/4
>>> print Mixed('-1-1/4') * 2
-2 1/2

You'll feel like you're in second grade all over again!

easy_install mixed to get it.

I hosted the code on Assembla, which has free mercurial repository hosting, so anybody who wants to polish it up is welcome to! hg clone http://hg.assembla.com/mixed_python mixed_python gets you a copy of the repository.

And all this because I was importing into a table last night and noticed some mixed numbers. A normal human being would have just converted them to floats in her head and gone to bed...

Sunday, February 24, 2008

Speaking at PenguiCon

I really enjoyed last year's PenguiCon - science fiction geekery and FOSS geekery make a surprisingly nice combination, with the SF people adding a nice lightheartedness to the atmosphere.

I'm giving a talk again this year, but this time, it will have nothing to do with Python - really! I'm going to try some of that hybrid vigor for my own talk, hoping that a fun theme will leaven a potentially dry topic.

Into the Dayta-b'hase: A First-level Introductory Adventure
April 19th, 2:00

Relational database technology underpins modern business, finance, engineering, and a lot of other boring stuff. You know you ought to learn something about databases, but how can you stay awake long enough? Come see database software used for something that actually matters: D&D characters. We'll set up a postgreSQL database, demonstrate all the fundamentals of relational databases, and advance Gruk the Unbathed to 4th level.

Northfield B
Hilton Detroit/Troy-Auburn Hills
5500 Crooks Rd
Troy, MI 48098 USA
(248)879-2100

Friday, February 22, 2008

how to spoil an OSS conference

I'm expecting to be in Washington D.C. on April 22-23, so imagine my delight when I learned of a "Military Open Source Software Conference" at that very time... in that very place! I work on a military contract! What could be better?

Now imagine my shock when I went to register. $1.945. For two days. Two days!

This, folks, is the problem with conferences run by for-profit training companies instead of volunteer organizations. No, wait, it's just one of the problems.
26 Senior Experts from: OSD, BTA, DHS, DON/CIO, AFAE, SPAWAR, FCS-BCT PO, HPCPO NGA, NASA/JPL, PEO-STI, USACE, Hill AFB, BAE, Boeing, Northrop Grumman, Lockheed Martin, Unisys, IBM, CSC, Rockwell Collins, Sun, Booz Allen Hamilton, Autodesk, SEI, and Mercury
What's wrong with this list? Not one representative of any open-source project! With a speakers' list like that, I don't know if it's even worth pilfering a hotel catering uniform and a pitcher of ice water to go eavesdrop on the talks.

Apparently, the topic is limited to decisions regarding open-source at a managerial level, rather than the actual use of the software and participation in its communities. That shows a serious lack of understanding. Breaking the separations between decisionmakers, code-writers, and users is part of what gives open source its power.

And suppose you, as an attendee, wish to suggest a topic, present a talk, or facilitate a discussion? Well, you can't. There's no provision for participation by the attendee community. Again, they very seriously do not get it, only seeing one-way relationships from paid content creators to passive content receivers. It seems that, just as "you can write COBOL in any language", you can buy proprietary software under any license.

It all makes me extra-grateful to the volunteers who run conferences like PyCon. (Register by March 7!)

Thursday, February 21, 2008

VC cloning into an existing directory

Mercurial won't let me clone into an existing directory.

This is a problem when trying to put (only) a subset of files under version control. For example, an Oracle installation includes an enormous directory structure. A few of its files really, really ought to be under version control (init.ora, tnsnames.ora, etc.); most shouldn't. I need an hg repository in the root of the Oracle software installation. Then, when I set up Oracle on a new machine, I need to clone that repository into the new machine's Oracle root... even though the new machine's Oracle root already exists and is chock-full of files.

$ hg clone src target
abort: destination 'target' already exists


I'm surprised that there's no --overwrite flag for hg clone, or anything like it, saying "go ahead and clone into that directory even though it already exists." Perhaps it's simply because VC programmers are software developers, used to working with a clean slate rather than configuring an existing directory structure? Anyway, I made up a script workaround.

hg clone $1 $1_clone_temp
cp -r $1_clone_temp/* $2/
rm -r $1_clone_temp


Or, in Windows,

hg clone %1 %1_clone_temp
xcopy /e /s /h /g /y %1_clone_temp %2
rmdir /s /q %1_clone_temp


Thanks to the Linuxchix irc channel for helping me figure out the syntax.

Dayton Oracle SIG

At last, it's time for DAY-O to meet!

Meeting of Dayton-Oracle SIG (DAY-O)

March 10, 2008, 5:30 pm
A hands-on, cooperative, laboratory-style meeting. Bring a wireless-enabled laptop if you have one. This meeting's topic will be Flashback.
Mary-Ann's Kitchen
33 N Ludlow St
Dayton OH 45402 USA
(937) 222-3663

Wednesday, February 13, 2008

Access forbidden. Attempted work detected.

The forbidden-content blocker at my worksite is more stupid than the one at your worksite. Doubt me? "Computing/Internet" is one of its forbidden categories. Blogs, another forbidden category, are the biggest problem because, as you know if you're reading this, most of the good code samples in existence these days are on blogs. We aren't allowed to appeal categories, only individual sites.

When I whined about it to my boss, he reminded me that I've done some work with web-scraping, and that I could probably write a Python script to make an exemption request for thousands of blogs at once, including ones I've never seen but might Google up someday. It'll benefit thousands of geeks across the organization, and maybe even convince somebody that they should turn the "stupid" dial down a notch.

I love my boss.

Here's the plan:
1. whitelist and blacklist of known legit and non-legit sites, respectively
2. count words. Words that appear much more frequently on whitelisted sites than on blacklisted are positive scorers, and vice versa.
3. apply the scoring vs. unknown sites. Hand-check a lot of results. Beef up the whitelists and blacklists and repeat until the scores against new sites are reliable.

Oddly enough, the hardest part of the process is gathering a long enough blacklist. I'm not just talking porn - I need harmless but non-work-related stuff, blogs where people are writing about their Irish setter instead of their SQL queries. (I need bunches of them - dozens, at least - so don't just comment to tell me, "My blog is useless and banal!")

One of the best sources I've found so far has been googling for "typical stupid blog".

Friday, February 01, 2008

TechFest 2008

If you're near Dayton, and you have a child, or know a child, or can find a child, or feel like a child, then you really need to get to TechFest!

Hands-on science and technology family fun
Feb. 16, 10 AM - 6 PM; Feb. 17, 11 AM - 5 PM
Sinclair Community College - Ponitz Center
444 W. Third St., Dayton, OH

I'm afraid the information technology people haven't traditionally brought stuff with quite the "wow" factor that the physical scientists and engineers bring. What's something we computer geeks could do that would compete with the chemists' goo-making or the physicists' pneumatic implosions? Have to think about that one.

in school, not even a Smurfette

So far, I've taken my smashing-planets introduction to Python to two groups of college students: the Wright State chapter of ACM and an OO programming class at University of Dayton. It's been lots of fun. I appreciate the chance to show young people something I really love, something I think can make their computer work more productive and more fun. My thanks to the folks who decided to invite me and the students who gave me their attention.

But. If I don't see a woman student in computer science soon - just, for crying out loud, one woman student - I am going to... I don't know. I'm going to have to research "how to go berserk".

What's going on in this country? I think I'd see better gender balance in a classroom in Iran.

Packet from table 6

I wish I knew more about networking. I have a technical need that I can barely describe, much less name.

For our new Dayton Oracle SIG, we want to run interactive, laboratory-style meetings. We'll meet in a place with wireless, one member will bring a laptop running an Oracle server and Oracle Application Server, and other members will use their laptops to access it and muck around.

This requires the computers using the wireless network to access not just the internet, but each other. I'd call this "peer-to-peer" networking, but it's really "peer-to-router-to-peer".

If I connect to my wireless at home and issue ifconfig, it reports eth1 ... inet addr:192.168.1.45. For other machines connected to the same router, that IP address is as usable as anything on the internet. If Oracle and Apache are running on the machine, I can use 192.168.1.45:1521 and 192.168.1.45:80 to log into the database or view webpages from other machines. If I go into my router and set Port Forwarding to ship incoming traffic to 192.168.1.45, then those become visible to the wider internet... but, to access them within the wireless network, I don't even need to do that. (In fact, I don't see any way I could turn it off even if I wanted to.)

So all we need is a meeting location that allows the same thing. It's not that simple, though. As far as I can tell, the wireless at Panera Bread (my first choice) doesn't permit this kind of inter-machine connection. That locally assigned IP address doesn't produce any response from other machines in the same network; it seems the router simply doesn't route traffic between machines. For security's sake, that kind of makes sense; I wouldn't want somebody hacking my database from the next table over. (Actually, if they did, I'd invite them to the SIG, but that's me.) But it removes some really neat possibilities for interaction.

I've found one place so far - the Hope Hotel in Fairborn - where the wireless does allow connections between machines. I was all ready to declare the hotel restaurant our meeting location, until my boss reminded me that it's full of GIANT TELEVISION SCREENS BLARING SPORTS and some people don't tune that out as automatically as I do.

I suppose there's always the option of bringing a router with me... but then I'd have to bridge people's traffic up to the internet somehow. I'm sure it can be done, it's just something I've never developed the skills for, nor do I own a particularly portable router. Hmm...

Thursday, January 24, 2008

Chicago

CODE Hacker for the World,
Programming Tool Maker, Stacker of w00t,
Player with Rails and the Nation's Error Handler,
SourceForge, Cheese Shop, CPAN,
City of the Big Dynamic Language Conferences:...

I just found out that Chicago, after hosting PyCon in March, will host YAPC::NA in June, and hosted RailsConf in 2006. Crazy!

Dave Barry's Only Travel Guide You'll Ever Need makes a joke that every single state claims to be "dynamic". Looks like they all lose to Chicago, though, because that's where the real dynamic types come to play.

Sunday, January 13, 2008

brilliant PyCon publicity ideas

I was alarmed to find that many of the attendees at CodeMash didn't know that PyCon 2008 will be in Chicago - right here in our beloved Midwest.

So, as volunteer publicity coordinator, I thought I should brainstorm on more ways to get the word out. Here's what I've come up with so far.
  • National "got whitespace?" skywriting campaign
  • Spread urban legend about programmer whose kidneys are stolen by organ smugglers when he fails to attend PyCon
  • Invite presidential candidates to debate their choice of Python web framework at PyCon
  • Post hilarious "DO NOT WANT STATIC MANIFEST TYPING!!1!" pic at icanhascheezburger
  • Vote Guido into final round of American Idol
  • Bribe fortune cookie factory employee to replace fortunes with "PYCON 2008 CHICAGO IN BED"
  • Use laser to engrave giant announcement in the moon's surface
  • JEN IN TEARS AFTER ANGELINA'S LIGHTNING TALK
Please supply more ideas in the comments... (serious ideas doubly welcome!)

CodeMash slides

Thanks, everybody, for another great CodeMash - even better than last year! More babbling about it will follow when I have time,

As promised, the material from my presentation. It's almost identical to my LinuxFest presentation of last September.

Saturday, January 05, 2008

Introducing cmd2

cmd is a Python Standard Library module for constructing command-prompt applications. It's very easy to use - just subclass its Cmd type, create an instance, and call that instance's .cmdloop(). To support a command, just write a do_yourcommandname(self, arg) method within your Cmd instance.

cmd is also very bare-bones. It forms the backbone of sqlpython, and while improving that package, I've created cmd2. cmd2 is an extension for cmd that adds helpful functionality without requiring any changes to cmd-based applications.
  • Searchable command history
  • Load commands from file, save to file, edit commands in file
  • Multi-line commands
  • Case-insensitive commands
  • Special-character shortcut commands (beyond cmd's "@" and "!")
  • Settable environment parameters
  • Parsing commands with flags

I've found several alternatives to cmd in the Cheese Shop - CmdLoop, cly, CMdO, and pycopia. cly looks wonderful, but I haven't been able to get it working under Windows, and that's a show-stopper for many potential sqlpython users. In any case, none of the alternatives are based on cmd - they're written from scratch, which means that a cmd-based app would need complete rewriting to use them. I like sticking close to the Standard Library whenever possible. cmd2 lets you do that.

Switching a cmd application to cmd2 is as simple as changing from from cmd import Cmd to from cmd2 import Cmd, and the new functionality is ready to go. See a usage example of cmd2.

It's in the Cheese Shop, so you can easy_install cmd2. (Cheese Shop page for cmd2)

Thursday, January 03, 2008

Geek Event Aggregator code

I've told several people that I would post the code for the Geek Event Aggregator "as soon as I get it cleaned up".

Today, I swallowed my pride and admitted that I've really got to post something - that the much-needed code cleanup isn't going to happen for months. So, I created a page for it at Google Code: http://code.google.com/p/whatwhenwhere/

Not that you want to do that. It's ugly. Really. It will be cleaned up someday, but if you really want to make use of it now, there it is.

I decided to name the project "whatwhenwhere" because the code might well be used to aggregate events that are not at all geeky, and to host it at Google Code because, well, I'd never tried it.

Most of the stuff you might be conceivably interested in is in scan.py, ship.py, and events/model.py.

Monday, December 17, 2007

CodeMash

Oops, I've been forgetting my blog badge.


CodeMash – I'll be there!


Bruce Eckel will be in charge of Open Space this time, which means it should be something else. Come with ideas for what you want to do with Open Space.

Wednesday, December 05, 2007

I promise not to make a habit of posting about xkcd


OK, I lied. But how could I have known this was coming?

Thursday, November 29, 2007

TurboGears and Oracle on OTN

Once, using TurboGears with Oracle was the kind of kooky idea you could only read about in the blog posts of obscure crackpots.

No more! The current top story at Oracle Technology Network is Daniel Rubio's article on using TurboGears with Oracle. Read, use, and enjoy! It's also got a permanent home at the OTN Scripting Language Technology Center.

Thanks to Oracle for all the great press lately, and to Daniel and Przemek for writing it!

Friday, November 23, 2007

PyCon: 141 submissions!

We weren't very good about getting word out for the PyCon call for proposals, and as the deadline approached, there weren't a lot of submissions in the queue. We worried about whether we'd messed up.

Oh, we of little faith! The last couple days before the deadline, the floodgates opened. We got 141 talks submitted! (And about 40 tutorials!)

And they're great submissions! If I had my way, I'd attend about 3/4 of them personally. Unfortunately, for a 3-day conference, we'll be lucky if we can fit half of them into the schedule. Well, I did suggest running PyCon on a 24-hour basis, but for some reason that wasn't accepted.

So now we've got to somehow narrow down the field to what we can fit in the con, and that's a painful process. Quite a few excellent talks won't go on the schedule simply because there's no room.

The good news is that we've got lots of space set aside for multiple tracks of Open Space sessions, which don't require any approval. So, in essence, we may get two PyCons running side-by-side - a traditional one with scheduled talks, and an unconference-like bunch of Open Space activity.

Too much good stuff... I'm definitely going to have to spawn some processes to see all I want to see.

Tuesday, November 20, 2007

New Python articles at OTN

There's a new series of articles on Oracle & Python over at the OTN Scripting Language Technology Center. Przemyslaw Piotrowski presents a detailed investigation of cx_Oracle, covering very nicely some aspects I'd glossed over in all my introductory articles. Great job, Przemek!

Microformat slides

Anybody who saw my Microformats talk, whether at the Dayton Web Standards Meetup or the DMA Linux SIG, can see my slides here or get a tarball here.

If you weren't at my talk, and are looking for a microformats presentation that will stand on its own, you're better off looking through the talk archives at the Microformats wiki. My talk isn't very self-explanatory without me standing there to talk about it.

Friday, November 02, 2007

CodeMash 2008

CodeMash is a great, multi-language agile programming conference here in Ohio. I had a great time last year, and next year looks even better.

Jan. 9 - 11, 2008
Kalahari Waterpark Resort
7000 Kalahari Drive
Sandusky, OH 44870USA
877-525-2427

The speaker committee has announced two sets of accepted abstracts for CodeMash 2008, with more still to come. Already, there's a lot for a Python fan to be excited about!
=============
Bruce Eckel: "Why I Love Python"
=============
Bruce Eckel has given hundreds of presentations throughout the world, published over 150 articles in numerous magazines, was a founding member of the ANSI/ISO C++ committee and speaks regularly at conferences. He provides public and private seminars & design consulting in OO Design, Python, Java and C++.

============
Leah Culver: "Getting Started with Django"
============
Leah Culver founded Pownce with her friends Kevin Rose and Daniel Burka as a way of sending messages, links, files and events to each other. Leah is the lead developer for the site, which has become one of the largest sites using the Django framework. Leah loves the challenge of developing a web application from scratch and writes about her experiences as a software developer at leahculver.com.

============
Kevin Dangoor: "Overview of the Dojo JavaScript Toolkit"
============
Kevin is the product manager at SitePen and the founder of the TurboGears open source web application framework. He has held positions in software development, management and sales engineering. He has previously spoken at CodeMash, PyCon, EuroPython and GLSEC and is the co-author of the book "Rapid Web Applications with TurboGears".

Kevin's talk isn't directly about Python, but I expect he'll cover how to tie Dojo into Turbogears. And if he doesn't offer it freely, I'm going to grill him on it!

Finally, my talk was accepted, too! It's basically a repeat of my Ohio LinuxFest "Introduction to Python" talk that used vPython to simulate a solar system, but since the audience this time will be a group of high-powered programmers, I should be able to amp it up and cover more ground. I'll try to dovetail with the more advanced talks at CodeMash so that (for instance) you'll be able to understand Leah's Django talk even if you arrived at CodeMash without knowing any Python.

There are also talks on Domain-Specific Languages, Silverlight (which, via IronPython, looks like a great way to Python-power webpages on the browser side)... and the speaker committee isn't even finished selecting talks yet.

Can. Not. Wait! Between this and PyCon in Chicago, it's truly an Ohio Pythonista's dream come true.

Python at MIT

There was a bit of buzz about a year ago when MIT announced plans to experiment with an alternative version of their electrical engineering/computer science (Course 6) curriculum - one that would start students with a Python-based course instead of the famous Scheme-based 6.001. It was exciting news, but I hadn't heard anything about it since then.

Today, a look through the department's webpage shows that the experiment has come quickly to fruition, and the new curriculum is the standard for this year's freshmen.

6.01, Introduction to EECS I
, with Python (and robots! Sounds FUN!)

I'm dying to know more about this! I'm going to hit up school friends for info, but maybe somebody here has more details to supply?

I still meet plenty of neckties who have never heard of any language besides C/Java/.NET (hey, it's the Midwest, and it's the Air Force), don't know why Python matters, and aren't interested in understanding its merits directly - only want to know "who's using it?". "MIT teaches it to all their EE/CS students" is a pretty good one-liner response. I'm not above leveraging MIT's name recognition. I did the time, after all (as a chemical engineer, not EE/CS, but still). Granted, it never won world domination for Scheme, but I bet it is the source of a very large fraction of the fame that Scheme does have.

Sunday, October 28, 2007

SQL*Plus alternatives (at InOUG)

I spoke last Thursday at the Indiana Oracle Users Group on "The Command Line Lives: SQL*Plus and Alternatives".

I still need to get the full, S5 format presentation posted somewhere; for now, you can see the raw text of the presentation here.

It's an interesting topic, and one I'd like to get drawn more deeply into. Some nice things have been done in producing open-source command-line alternatives to SQL*Plus, includingUnfortunately, the only one whose development currently seems to be active is sqlpython, and there's just me and Luca Canali working on that. I've just added functionality to sqlpython to let it fail over to sessions of the other tools (when "set failover true" is issued; doesn't work on Windows). That way, the exciting features of Senora and YASQL can be used from within sqlpython, instead of trying to remember which tool has which feature. It's quirky, though, since a full-fledged independent session of each tool is actually maintained "under the covers".

I'm dreaming of expanding sqlpython to the point where it incorporates all Senora and YASQL's features natively. Tragically, my dreams cannot yet be downloaded from the Cheese Shop.

Wednesday, October 10, 2007

xkcd

I promise not to make a habit of posting about xkcd.

But only because I'm counting on you to read it daily yourself.



I printed this today and then wandered the office, looking for someone to show it to, gradually coming to the horrifying realization that I don't work with anyone who would get it.

Tuesday, October 09, 2007

cx_Oracle and Oracle XE on Ubuntu

When I first posted about using Oracle XE on Ubuntu, I thought that getting cx_Oracle working with XE was pretty easy. I was wrong! Last night, I installed on a "clean" machine, and it was a lot harder; perhaps last time, I benefited from packages I'd already installed, meeting secret prereqs without knowing it.

[EDIT: As of June 2008, a streamlined set of instructions is here.]

Fortunately, I did work out the necessary steps.
  1. sudo apt-get install libc6-dev

    If you don't, you'll get
    In file included from /usr/lib/gcc/i486-linux-gnu/4.1.2/include/syslimits.h:7,
    from /usr/lib/gcc/i486-linux-gnu/4.1.2/include/limits.h:11,
    from /usr/include/python2.5/Python.h:18,
    from cx_Oracle.c:6:
    /usr/lib/gcc/i486-linux-gnu/4.1.2/include/limits.h:122:61: error: limits.h: No such file or directory
    In file included from cx_Oracle.c:6:
    /usr/include/python2.5/Python.h:32:19: error: stdio.h: No such file or directory
    ... when you try to install cx_Oracle.

  2. export ORACLE_HOME=/usr/lib/oracle/xe/app/oracle/product/10.2.0/server/
    export LD_LIBRARY_PATH=$ORACLE_HOME/lib
    export PATH=$ORACLE_HOME/bin:$PATH


    If you don't, you'll get
    error: cannot locate an Oracle software installation
    or

    ImportError:libclntsh.so.10.1:cannot open shared object file: No such file or directory
  3. Download the source code tarball, unzip and unpack, cd into it, and
    sudo python setup.py install
    You will get
    error in cx_Oracle setup command: Oracle home does not refer to an 8i, 9i, 10g or 11g installation
    because setup.py doesn't account for XE. You need to hack it.

    Download the patch useXE.diff and run
    patch setup.py useXE.diff

    Now run
    sudo python setup.py install
    again.

    I'm going to submit a patch to the cx_Oracle folks, so hopefully it will be fixed for the next release.

    [EDIT: Anthony Tuininga, cx_Oracle's developer, has taken my patch, improved it, and incorporated it into the cx_Oracle trunk; as of the next cx_Oracle release beyond 4.3.3, this is fixed. This sort of thing is what makes open source fun!

    To grab the patched version from the cx_Oracle development trunk right now, install Subversion on your machine, then
    svn co https://cx-oracle.svn.sourceforge.net/svnroot/cx-oracle/trunk cx-oracle
    ]

  4. If you're accessing Oracle XE on your own machine, start your listener. On my machine, at least, the XE installation didn't do that.
    sudo su - oracle
    export ORACLE_HOME=/usr/lib/oracle/xe/app/oracle/product/10.2.0/server/
    export LD_LIBRARY_PATH=$ORACLE_HOME/lib
    export PATH=$ORACLE_HOME/bin:$PATH
    lsnrctl startexit


    You may want to edit your .bashrc to do this; otherwise, you'll need to for every session in which you access Oracle.

    If you're using Oracle XE as a client to access Oracle (XE or otherwise) on another machine, skip this step, but you'll need to make sure the SID you're looking for is in $ORACLE_HOME/network/admin/tnsnames.ora.

  5. At last, you can start using cx_Oracle! See the DB-API2 Cheat Sheet for some quick syntax reminders.

EDIT: comment from teambob

I mistakenly deleted the following comment:

I installed via easy_install, but there was a bit of a trick.

After setting the environment variables I installed using the following command: sudo -E easy_install cx_oracle

The "-E" option to sudo causes all your environment variables to be passed to easy_install

Saturday, September 29, 2007

Ohio LinuxFest slides & code

Thanks to everyone for your attendance and feedback at my OLF talk! I had a great time!

A tarball with everything you need to duplicate my presentation is here.

It's pretty complex, so it includes a README file. (I used s5 slides generated by rst2s5, a Socrates file from Bruce, and eight directories of code files. Don't worry, though, it's very lean - just 60K.)

Monday, September 10, 2007

I'm speaking on Python at OLF

What's this? You haven't registered for LinuxFest yet?

But I'm (absurdly honored to be) one of the speakers!

Introduction to Python and/or Solar System Demolition
An introduction to Python programming. Demonstrates the language by developing a graphical solar system gravitation simulation in real-time.
September 29th, 2007, 3:50 - 4:40 PM
Greater Columbus Convention Center
400 North High Street
Columbus, OH 43215 USA

And I'm bringing my own projector this time, just in case... (plus I've got nVidia drivers installed, which should help...)

Ohio LinuxFest: register now!

Drop everything and register for

Ohio LinuxFest


It's free - but the registration deadline is Sep. 14 - this Friday! Move, move, move!

Ohio LinuxFest
September 29th, 2007
Greater Columbus Convention Center
400 North High Street
Columbus, OH 43215 USA

Thursday, August 30, 2007

Microformats talk

Have I mentioned that the Dayton Web Standards Meetup is a great group? It's brimming over with good ideas for every sort of web writer. Very highly recommended!

This coming Thursday, I'll be speaking on Microformats! Microformats are a wonderful way to make your information available to automated tools, while mandating nothing about what else you do on your page. They're great fun, they're catching on, and I'm looking forward to it!

5:30, Sep. 6: Microformats talk at Dayton Web Standards Meetup
at Panera Bread on 1203 Brown Street.
See you there!

Okay, I admit, I wrote this post partially to practice writing a microformat. Install Operator into your Firefox. Now, with one click (each), you can grab a vCard for Panera, get a map there, or pop the event inot your calendar program. Oooooh!

Wednesday, August 29, 2007

Dayton-Oracle is for real

The Dayton-Oracle mailing list I set up a couple years ago is bearing fruit. The Dayton Oracle User Group is real! We live! We had an organizational pre-meeting last night with five great people who were willing to take up the burden of making this happen.

We had a lot of rich discussion about the way we want to approach running the group. In many ways, we're going to break from the traditional format and content of Oracle User Groups and adopt some practices better-known from the open-source world: more interactive, user-driven content; zero-cost bring-your-own-food arrangements instead of catered lunches with meeting fees and sponsored content; evening rather than daylong meetings; etc. I think these are good practices on their own merits. Also, there are already good traditional user groups in Columbus, Cincinnati, and Indianapolis, so if we want a successful niche between them, it makes sense for us to differentiate; we're not likely to be a better OOUG than OOUG, after all. I hope Dayton people who do already attend OOUG or GCOUG will continue to do so, and will enrich themselves yet further with our meetings.

Watch this space for further announcements, including our first meeting time (sometime early October) and a link to our super-awesome website (once it's built).

Thursday, August 16, 2007

openSUSE and the bleeding edge

I fear I'm about to sound both boastful and ungrateful... but my boss refreshed my work laptop with this monster I have dubbed "DellZilla". Cool, huh? Definitely - except that you need a pack mule to actually move it. Once upon a time, laptops could actually fit in your lap.

The other problem is that I haven't found a Linux distribution that comes with the right drivers for it. Ubuntu Feisty was hopeless - all incarnations (u-, ku-, and xu-) were flummoxed by the screen and failed to start xwindows, or gave an utterly black empty screen.

Next I tried openSUSE, and was in some ways pleasantly surprised. It copes OK with whatever video voodoo is required for this monster screen. It automatically noticed that I had a Windows Vista partition and included it as a boot option. In fact, it includes "Restart and boot directly into Windows" as a menu option, so you can go get a cola while switching OS's instead of hovering over the keyboard waiting to catch the boot loader. Considerate! And it comes with all the cutting-edge goodies like Beagle. (And freeciv! Eek!)

One significant complaint is that, if you don't have wireless connectivity while you're doing the openSUSE installation, YAST won't even record the URLs of the repositories you'll need for software. When you need to download the good stuff later on, you'll need to hunt around for the URLs (psst, they're here) and add them to your list of sources by hand. For a SuSE newbie, that's a pain.

But I got through that. The real problem is that I've got no support for the mac80211 wireless card. Trying to get a driver installed for that has been an adventure in frustration. There's a driver here, but it appears to my amateurish eye as though YAST installs the Linux kernel-source into locations where the mac80211 makefile absolutely does not expect it, and trying to hack them into agreement is a step too far for me. It's like being back in the Bad Old Pre-Ubuntu days when Linux was something I only wished I could use.

So here I am, typing on a zillion-dollar laptop and tethered to the wall like it's 1994, and trying to figure out my next move. This says Sabayon 3.4 supports mac80211. I hadn't even heard of Sabayon before this week, and I've read good things about it, but I wonder if it's just too fancy. It installs from DVDs; I've never even tried burning one of those. And I can't even get its blasted homepage to come up under Firefox or Konqueror. But whatever; I need wireless. If you don't hear from me for a while, it's because I need to compile my own drivers for the keyboard, or maybe the AC adapter.

Tuesday, July 31, 2007

birthday list

Pardon my long absence from the blogosphere. One cause has been that my workplace blocked access to Blogger, all Blogger-hosted blogs, all mailing lists anywhere, etc. I'm becoming thoroughly frustrated. The message it seems to send is, "We don't desire the fruit of your work as much as we value our infantilizing mistrust of your professionalism. We pay for the services of skilled adults yet believe that we got delinquent children instead."

But anyway. That's not why I'm posting. I've been fairly nomadic lately, migrating my work from machine to machine and experimenting with various operating systems, and it's made me crave some good organizational software. Since my birthday is coming up, I thought I would ask the Universe to produce it and see what happens.

1. I need help keeping track of the difference between files I've worked on - stuff I want to keep when I move or rebuild - and stuff I can leave behind because I just downloaded it from somewhere and I'll just download it afresh on a new system anyway. Then it'll be easy to back up and transfer my work. This software will note when I modify a file with a text editor, IDE, image editor, OpenOffice, or whatnot, and will flag it as my own personal work - a "keeper". Stuff that comes from downloads, from expanding zipfiles, automated processes and loggers, etc. will all be flagged "ephemeral".

2. I need help reorganizing my folder structure. I'd like something that can scan my drive and notice that my photos are scattered in four different places, short text notes are all over, I've got four copies of the same file in four directories under three names, etc., and propose and facilitate a consolidation plan.

3. A nice tag-based file access system That could actually replace #2 and be a key component of #1.

If you know of this stuff already existing, speak up! If you want to write it, hooray! If you want to wait several years until I have the skill and the time to do it myself, well... keep reading my blog, I guess. If you are saying, "You wouldn't need any of this if you exercised organizational discipline when creating your files", then please note that I am Sticking My Tongue Out At You.

Friday, June 15, 2007

get a job, sha na na na

If you're reading this and you're interested in switching jobs, let me know...

... because I'm constantly being asked these days whether I know people looking for work. It's a good feeling, but it would be a better feeling if I had somebody to refer them to. And I'm sure I'm not the only one getting these inquiries, so if you've felt stuck writing boringCode for a pointy-haired boss, it's a great time to let your friends know.

This applies to Python people as well as Oracle. That's a neat change; when I got involved in Python a few years ago, the assumption was that most Python programmers had day jobs using more boring languages, and could only use Python in their passion-driven midnight coding sprees. Now, employer interest is very high - it seems employer interest has actually grown faster and overtaken the population of Python coders. That's especially noteworthy when you consider how quickly someone can become a Python programmer.

Anyway, good times for Py and Ora geeks. Woo hoo!

Thursday, May 31, 2007

Kubuntu

For Feisty, just to learn something new, I thought I'd try Kubuntu (KDE Ubuntu) instead of Ubuntu this time around.

And... and... I think I'm a believer.

For what I've done so far, the extra information KDE provides is very well thought-out, just what I need to know when I need to know it. For instance...
  • When I want to remove a USB drive or somesuch, but something is still using it, Ubuntu (like Windows, including Vista) only tells me that something is using it. Kubuntu tells me what is using it.
  • Alt-Tabbing through my open windows gives more information than just the icon of each running program - enough so that I can distinguish between my eight different command prompt windows.
  • Hovering over an icon pops up the file properties. Nice! I mean, why else would I be hovering over an icon, except that I was fumbling for the right-mouse button so I could find out more?
  • Clicking my networking icon provides all the information I need to choose at a single glance, and then one more click connects me. Also, Kubuntu seems to jump to conclusions and do the right thing for networking - when there's only one wireless network around, Kubuntu just connects me without waiting to be asked.
The downsides I've found so far...
  • I really liked Ctrl-+ to increase the font size of GNOME windows. That's important when trying to read laptop screens in adverse environments (like outside). I haven't found a KDE equivalent.
  • The built-in text editor, Kate, bungles the spaces/tabs in Python files. That's one good reason to install Komodo (free editor or the real thing) immediately.
Real gurus can discuss the relative strengths of KDE and Gnome for hours, of course, and I thank God that I've never been caught in such a debate. I'm just saying that KDE is making a really good first impression on me, and that's after years of Gnome experience.

Friday, May 11, 2007

Extensible CheckBox List

Introducing a brand-new widget for TurboGears: Extensible CheckBox List!

To use it:
  1. easy_install xcbl
  2. In your controller.py,
    from xcbl.widgets import ExtensibleCheckBoxList
  3. In your form's WidgetsList,
    pets = ExtensibleCheckBoxList(options=['cats','dogs','turtles'])
    (exactly the same as you would for an ordinary CheckBox List widget)
The result - a CheckBox List with an "other" field that lets the user hand-enter new options - as many as they want.

Unfortunately, the new items will appear beneath the "Other:" box, not above it, unless you upgrade your MochiKit to 1.4. Details on how to do that and more are here.

Developing it was lots of fun - it really shows how easily a relative amateur can contribute to TurboGears. I hope to write up a "Developing and Contributing TurboGears Widgets for Dummies" document soon, based on the experience.

cleansing table and column names

What kind of joker thought "BASE POC'S" would make a good column name? Sheesh, MS Access. (Not to mention apostrophe abuse. See Bob's Quick Guide to the Apostrophe, You Idiots.)

To my surprise, I googled "script rename invalid columns oracle" and got nothing useful. So here's a chance to play with Oracle regular expressions.

You may be thinking, "If they're illegal column names, how can they get in the database in the first place?" With "quotation marks", that's how. In fact, you can
CREATE TABLE "!!!My Rockin' Table!!!"
( "*^@(@ @!!!" VARCHAR2(30)
CONSTRAINT " -- d00d --" PRIMARY KEY,
"miXiNG CASE is TEH KEWList" VARCHAR2(30) );
and, not only will all your hideous names be used verbatim, but they will be case-sensitive. If you take advantage of this feature, you should be slapped. I guess the reason Oracle makes this possible in the first place is so that an import can be accepted from a source - like MS Access - regardless of how awful its names are.

Anyway, here's a PL/SQL cure for bad table and column names.
(link to code)
/*
Renames tables and columns to be Oracle-legal.

SET LINESIZE 200 before running to avoid unwanted page breaks.
Run from an account with privileges to
- access v$reserved words
- alter the appropriate tables

Thanks to Eddie Awad for his description of v$reserved words:
http://awads.net/wp/2007/01/10/what-does-reserved-y-really-mean/

Catherine Devlin, catherinedevlin.blogspot.com, 11 May 2007, v0.1
*/
CREATE OR REPLACE PROCEDURE cleanse_names IS
new_obj_name VARCHAR2(30);
CURSOR tbl_names IS
SELECT owner,
table_name
FROM all_tables
WHERE dropped = 'NO';
CURSOR col_names IS
SELECT owner,
table_name,
column_name
FROM all_tab_columns
JOIN all_tables USING (table_name, owner)
WHERE dropped = 'NO';

PROCEDURE do(this IN VARCHAR2) IS
BEGIN
DBMS_OUTPUT.PUT_LINE(this || ';');
-- Uncomment if you're feeling decisive
-- EXECUTE IMMEDIATE(this);
END do;
FUNCTION legalize(name IN VARCHAR2, replacement_letter IN VARCHAR2)
RETURN varchar2
IS
new_name VARCHAR2(30);
is_reserved NUMBER(11,0);
BEGIN
-- must begin with a letter
new_name := REGEXP_REPLACE(name,'^[^[:alpha:]]',replacement_letter);
-- and contain only letters, numbers, #,_,$
new_name := REGEXP_REPLACE(new_name,'[^[:alnum:]#_$]','_');
-- and mixing case is evil
new_name := UPPER(new_name);
-- and we must respect Oracle reserved words
SELECT count(*)
INTO is_reserved
FROM v$reserved_words
WHERE keyword=new_name
AND (reserved='Y' OR res_semi='Y');
IF is_reserved > 0
THEN
new_name := SUBSTR(new_name, 1, 29) || '_';
END IF;
RETURN new_name;
END legalize;
BEGIN
DBMS_OUTPUT.PUT_LINE('set scan off');
FOR col IN col_names
LOOP
new_obj_name := legalize(col.column_name, 'C');
IF new_obj_name != col.column_name
THEN
do('ALTER TABLE ' || col.owner ||
'."' || col.table_name ||
'" RENAME COLUMN "' || col.column_name ||
'" TO ' || new_obj_name);
END IF;
END LOOP;
FOR tbl IN tbl_names
LOOP
new_obj_name := legalize(tbl.table_name, 'T');
IF new_obj_name != tbl.table_name
THEN
do('ALTER TABLE ' || tbl.owner ||
'."' || tbl.table_name ||
'" RENAME TO ' || new_obj_name);
END IF;
END LOOP;
END cleanse_names;

Larry Wall, God, Lois McMaster Bujold

Part of what got me thinking about my last post is that I'm suffering a little crisis over Larry Wall. I'm really admiring his spirit. For instance, I like his way of expressing his Christianity.

As if that weren't enough, he's into The Lois, as quoted here:
Note how we still periodically hear the phrase "serious literature". This is literature that is supposedly about Real Life. Let me tell you something. The most serious literature I've ever read is by Lois McMaster Bujold. Any of you read her? It's also the funniest literature I've ever read. It's also space opera. "Genre fiction," sneers the Modernist. Meaning it follows certain conventions. So what? Nobody in the world can mix gravity and levity the way Bujold does in her Vorkosigan books. It's oh so definitely about real life. So what if it follows space opera conventions. Sonnets follow certain conventions too, but I don't see them getting sneered at much these days. Certainly they were always called "serious".

How long till Bujold becomes required reading in high school? Far too long, in my opinion. Horrors. We wouldn't want our students actually enjoying what they read. It's not--it's not Real Life.

As if the Lord of the Flies is real life. Feh.
Yes, I'm a Lois fan, too. I once drove hundreds of miles to an SF con purely because she would be there...

So, anyway. Yaaaay, Larry. But there's no getting around this: I prefer Python. Perl is great, Ruby is great - all dynamic languages contribute to each others' success, I think, because the prominence and respect won by each rubs off the others, so we can get our necktie-level blessing. But, when it comes down to what I use, I can't write anything else these days without wishing it were Python.

Tuesday, May 08, 2007

getting a bit philosophical

Have you had the experience of struggling with a problem for hours, going to ask for help with it, and then realizing the answer to the problem while you're in the middle of asking the question?

Of course you have. It's an amazing phenomenon. It really shows how much the brain depends on language to think clearly. It's one reason I'm such a junkie for user groups and conferences! And it's not good enough to just hear it or read it - you have to speak it or write it to really engage your brain. Similarly, sometimes it seems like you never really understand a subject until you have to teach it to somebody else.

Meanwhile, it's become an axiom of conversational politeness that politics and religion are to be avoided in most contexts. Which means... which means that we're forbidding ourselves to use our most powerful means of stimulating clear thought... for some of the most important ideas we have to think about. Hmm...

Wednesday, May 02, 2007

Microformats

As I work more on the Geek Event Aggregator and similar projects, I'm getting really fascinated by movements like microformats and POSH - efforts to make web content more semantically meaningful and machine-readable, without trying to radically reinvent the web. Just small tweaks to the HTML we've been writing all along can make a world of difference - indeed, often just using HTML best practices that have been around (and largely ignored) forever. Interesting stuff! I'll probably be blogging about it more in the future.

One related concept is blogchalking, so here goes.
This is my new blogchalk:
United States, Ohio, Dayton, Brookville, English, German, Catherine, Female, 31-35, Python, environment. :)
That's for machines to read. Humans, avert your eyes.

Let's see, I have about three more months before I have to update that age category.

Wednesday, April 25, 2007

Unicode error with TurboGears and Oracle

Thanks to kap_ravi's comment, I tracked down a surprisingly big and poorly-documented stumbling block for folks using TG with Oracle.

I've updated the original TG+Oracle blog post to reflect it, but the gist of it is that you're likely to get
SQLError: (NotSupportedError) Variable_TypeByValue(): unhandled data
type unicode
errors unless you include
sqlalchemy.convert_unicode = True
in your .cfg files.

Configuring Oracle for proper Unicode use may avoid the need for this; I'm not sure. My dreadfully mononational working environment has prevented me from developing any Unicode experience.

Monday, April 23, 2007

good errors vs. bad errors

One of the things I love most about PostgreSQL is its beautiful error messages. For instance, let's say I've inexplicably forgotten to add my home state to a table of states, thus violating a foreign key constraint when I try to insert a Minnesotan city. Here it is in Oracle:

SQL> insert into city values ('International Falls','MN');
insert into city values ('International Falls','MN')
*
ERROR at line 1:
ORA-02291: integrity constraint (MYSCHEMA.FK_CITY_STATE) violated - parent key not found

OK, that doesn't suck. In this case, it's pretty easy to figure out the problem, especially since I gave my constraint a meaningful name. However, there are cases where the foreign key is not so obvious - maybe it involves multiple columns - and I'm sometimes sent off to look up details on what FK_CITY_STATE involves. Since DESCRIBE won't work on a constraint, that means either an intricate query joining several V$ tables, or a graphical tool like Toad or SQL Developer.

Now I'll make the same mistake in psql.

mydb=# insert into city values (2,'International Falls','MN');
ERROR: insert or update on table "city" violates foreign key constraint "fk_city_state"
DETAIL: Key (state)=(MN) is not present in table "state".

Now that's clarity! It's one of the things I love about PostgreSQL. After all, the database has all this detail easily accessible to it; why shouldn't it deliver it to me along with the error message? It's even more important when an error like this is thrown from somewhere deep inside some code, where it may not be obvious to me which statement or which row is responsible for it - having the offending value displayed is incredibly convenient. It's considerate programming - no more, no less.

I'll say this for Oracle, though - Oracle's standardized ABC-12345 error codes are really handy for looking them up, for searches on Metalink or Google.

In any case, either of these SQL environments have good error reporting. But what about Oracle Enterprise Manager?


Where am I supposed to go with this? Shall we go search on Metalink for "An error has occurred"? Does someone at Oracle imagine that this message is, in some mysterious way, useful? Maybe there's some logfile somewhere on the server that will tell me more... maybe... somewhere. Maybe. Somewhere.

Seriously, folks. OEM has been around for a long time now. It still throws lots of errors, often for no apparent fault of the DBA (I got this one simply by trying to access the "Administration" tab.) But I would forgive throwing lots of errors, if the errors were clear, and if they included information to facilitate finding out more. This, however, I cannot forgive, and it is all too typical of my experiences with OEM. (Well, a few years back, the errors tended to be screenfuls of Java error-stack vomit; now most of those seem to have been "cleaned up" into these poker-faced non-messages. That's progress? It's like Windows 3.1 all over again.)

So, if you see me rolling my eyes next time an Oracle rep shows a slide telling us how wonderful lovely OEM makes DBAs' jobs so much easier... this is why.

Friday, April 20, 2007

Hello Penguicon

I'm having a fantastic time at Penguicon so far...

except, of course, for being unable to connect my laptop to the projector for my talk. Aaaaaaaaaah!

If you were in my audience, 1) thank you for bearing with me, 2) I hope you managed to get something out of it, and 3) I'm so glad you came here!

The full tarball of talk material is available here. You can use it to look through the plain-text version of the talk (independently available here), or install vpython on your system and run the demos (I recommend running 'python solarSim4.py', then looking at the code), or actually install Bruce on your system so you can run 'socrates.py pyIntro.soc' and see the presentation the way you should have seen it at the con.

Thanks for being a great, involved - not to mention forgiving - audience!

Thursday, April 12, 2007

Excel reports from TurboGears

Last week, I had the chance to design, build, and deploy a TurboGears application to outside customers for the first time ever. It was awesome! It was a simple survey form - well, simple in concept, but various questions were dependent in various ways on the answers to other questions, so I did have to get into the JavaScript, praise be to MochiKit.

The application owner wanted to see his survey results in (hmm, can you guess?) MS Excel. "No problem", I said, and then was surprised at how much trouble I had figuring out how to do that straight from TG. AFAIK, TurboGears doesn't have a handy way to output CSV. I could certainly produce a webpage with CSV data, and even use the default method in my controller to serve it with a .CSV extension ("http://myserver/report.csv"), but it still had webpage-type headers and thus Excel still didn't know what to do with it. And no, I wasn't about to tell him to cut-and-paste.

What I eventually hit on was to provide the results as a simple HTML table, then take advantage of the fact that Excel can open a webpage that contains (only) a table rather nicely. Then I gave him a batchfile that simply said

"c:\Program Files\Microsoft Office\Office11\excel.exe" http://myserver/report

... double-clicking it gets a live view of the data.

If he'd been cooler, the batchfile could just as well have said
"c:\Program Files\OpenOffice.org 2.2\program\scalc.exe" http://myserver/report
- I tried that, too, it works.

Actually, I didn't serve it under http://myserver/report, I served it under https://myserver/reportWithHideousGUIDblahblah4242rtfm22hike. That was my quick-and-dirty way to provide some basic security - the report URL is unguessable, so unless he shares the link, it should be safe. On the other hand, anybody who can sniff his request can extract the magic URL and use it. Can anybody comment on how much of a risk that is? There's no real sensitivity to the data in this case, but it would be nice to know if this the-URL-is-the-password scheme is a worthwhile shortcut or is terribly dumb.

Tuesday, April 03, 2007

Presenting at Penguicon

Well, two weeks from Friday, I'll be presenting on Python in Michigan... wearing a Starfleet uniform, if the vendor area opens in time.

Yes, I'll be at Penguicon. It's an open-source convention - it's an SF convention - it's a dessert topping! Troy, MI, April 20-22. I've never been there before, and I expect to enjoy it. The schedule included two higher-level Python talks, so it seemed to me like somebody should introduce Python so newbies could get the background they need to absorb the higher-level stuff. Wish me luck!
Introduction to Python, April 20 7:30 - 8:30 PM, Catherine Devlin

Django the Python framework for rapid web app development, April 21 9:30 - 10:30 AM, Dan Scott

Patterns of Data Access implemented in SQLAlchemy, April 21 6:00 - 7:00 PM, Mark Ramm
I also hope to make it to
PostgreSQL & Explaining Explain, April 20 10:00 - 11:00 PM, Aaron Thul
and to sit on the Constructed Languages panel (I'm an Esperantist)... and not to sleep, apparently.

My presentation will be more or less a repeat of the one I gave at Dayton Perl Mongers - it went well there, and the planetary theme should be even more appropriate for Penguicon. In fact, now that I've got a good talk to give, I'm starting to look around for any old place, any old audience in the area I can introduce Python to. Suggestions are welcome! centralOH@python.org is gaining membership pretty nicely, but we're still going to need more to turn it into a really thriving in-person group. Hopefully I can lure a bunch of new people into the joys of Python.

Monday, April 02, 2007

Feeling the love from Oracle

The Oracle Technology Network has rolled out two new features deeply gratifying to the PyOraGeek...Thank you, OTN! This attention is important for more than just the benefits to those of us already hooked on Python. Oracle's interest carries a lot of weight; when Oracle embraced Linux several years ago, it was a big step forward in establishing Linux's credibility in the corporate world. I think we can expect a boost, too.

Wednesday, March 14, 2007

Intro to Python presentation

I've put the presentation for tonight's Intro to Python talk here.

It's not exactly the "slides" - it's a Bruce presentation. Bruce is great for live demos, especially for Python. Unfortunately, there's no "slide deck" that you can review statically - if you want to use it, you have to install Bruce and run it. The TAR I posted has a README with some basic instructions.

Since many conference organizers want a slide deck, one nice addition to Bruce would be a way to generate a slide deck. (No, I'm not volunteering... yet.) The other wart I ran into was the difficulty of changing the font sizes for the interpreter. Setting values in Socrates !CONFIG directives and config.py didn't work; I had to dig down into baseinterp.py and hard-code new font size values. That one, I probably could patch, if I get time.

Anyway, I did end up using Visual Python to do some fun solar system simulations. That should hold audience attention, I think...

Thursday, March 08, 2007

Dayton, meet Python. Python, Dayton.

I'm giving an introduction to Python at the next Dayton Perl Mongers meeting:
(Caution: the hCalendar creator didn't work for me with Opera or IE, only Firefox.)

If you're in the area, and still haven't learned Python (how have you put up with this blog?), please drop by!

I think that, to keep some visual interest up, I'll use Python's turtle module for some of my demonstrations. Not exactly snooty enterprise stuff, but turtle graphics are cute. Either that, or possibly vPython, which is certainly more impressive - but a bit more complex and less intuitive, possibly wrong for an introduction.

(I haven't forgotten my promise to post about PyCon. I'm waiting for presentation links to be published so I can refer to them.)

Monday, March 05, 2007

Geek Event Aggregator design notes

I finally created a central page for the Geek Event Aggregator - mostly it's design notes, but it's also a home less ephemeral than a bunch of blog posts:

http://catherine.devlin.googlepages.com/geekEventAggregator.html

Monday, February 26, 2007

Thanks to Google

... the Geek Event Aggregator is working again. Shortly after my post about the upload to gCal not working, I got an email from Ryan Boyd, of Google's Calendar team:
Someone brought the following post to my attention and I thought I'd follow up and see if I can be of any assistance... The problem that you are experiencing (500 error: Cannot access the calendar you requested) is due to the large size of (number of events in) the calendar.
Wow! Thanks to whoever passed my whining on to him... let me never speak ill of this social web stuff!

With that information, I was able to get it working again, so go ahead and check out that APL conference near your uncle in Kalamazoo.

Better yet, I was able to give my Lightning Talk on it, and it was very well received. The Lightning Talks are so popular that PyCon doesn't even schedule anything opposite them anymore, so basically the entire conference body of 593 people (!!!) was there; I've never spoken before so many in my life. I'm glad that didn't occur to me until afterward.

In the talk, I briefly outlined the architecture. At some point I'll post about it here. The basic principle is: HTML interpretation is hard, let's go datetutils.parser.parse()ing!

Yeah, PyCon! I just got back, and I've got so much to say; expect a flurry of blog posts as soon as I can get to it. I'm as happy as a clam. As several clams! As several unusually happy clams!

Thursday, February 15, 2007

Geek Event Aggravator

I swear to you, the code that gathers data for the Geek Event Aggregator is running great.  It's harvesting over 5,000 events, with ever-improving completeness and accuracy.  

Unfortunately... the code that ships the harvested data up to Google Calendar - which worked perfectly at first - is now falling flat, with gCal returning 
"Error: Cannot access the calendar you requested." 
when I try to ship the new data up, or to clear out the old data - thus the redundant entries.  (No, it's not a simple password problem, I did check that...) 

I suspect that I triggered something at gCal that said, "Hey, there's way too much activity on this calendar, it must be some evil bot, lock it down."
  
I guess this is the problem with mashups.  It's great to springboard off the work of total strangers - but you're at the mercy of a black box.  Maybe I can buttonhole somebody from Google at PyCon next week (holy cow!  PyCon next week!) and ask for a contact somewhere in the bowels of the gCal team who might be willing to help me troubleshoot.  The sad thing is, this pretty much sinks my plan to do a Lightning Talk on it.

So anyway, my apologies if you've been trying to use the Aggregator and have found only stale and redundant information.  My box is overladen with fresh, hot event data; I just have to figure out a way to get it out to you.  I tried Swivel, but can't seem to get the data there, either.  And the old-fashioned solution - the app running in the Oracle Application Express sandbox - can only receive data by manual uploads, and then only in small chunks; I would have to do about fifty mouse clicks every time I wanted to refresh the Aggregator's data, and I'm sorry, but I'm not a clicker.

The ultimate solution, of course, is to build my own app on my own web server, or on a web host with permissive policies on installing software.  I don't have those resources right now, unfortunately.  (sigh)

[I apologize; Blogger has gone loony on the line breaks for this post, and I can't seem to fix it. Grumble. All my web resources are letting me down today.]

Sunday, February 11, 2007

Presenting at Dayton-Cincinnati Code Camp

The following topics you submitted have been selected to be included in this year’s Code Camp:

· Data in Python, from ASCII to ZODB

Yay! Be in West Chester, OH (i.e., Northern Cincinnati, basically) on March 24 for a fun, free day of Code Camp. Most of the content is .NET, but they're very gracious about encouraging other content, too.

My talk is going to be a broad survey of ways you can get data of various sorts into and out of Python programs - not just RDBMS, but everything from the simplest to the most elaborate of data infrastructures. It's a lot to take on in an hour, but I think it'll be good for raising awareness of the breadth of possibilities - even for non-Python programmers (you poor things). I know I've learned quite a bit already just outlining the talk.

Thursday, February 08, 2007

held for ransom by Vista

I attended an MSDN event here in Dayton last fall, where I was given a beta CD of Vista. I finally got around to putting it on my Windows partition last month. The install went pretty smoothly - it had an option to upgrade the XP partition, preserving the applications and data already present.

I didn't really play with it long enough to form a firm opinion on Vista, but the experiment ended rather badly. Vista popped up a window telling me that the trial period had expired, and it was time to pay up. I had to go to work at that point, so I shut down the machine, brought it into the office, and booted it up again. Then, the "trial period expired" window and a browser window - supplied so that I could buy a license at the MS site - were the only functionality that Vista would enable.
I couldn't go to my desktop, Windows Explorer, a command prompt, or anything that would enable me to access my hard drive. Files that I had created under my paid-for XP were now inaccessible to me; no way was provided to copy them off before reinstalling XP. My hard disk was, in essence, being held hostage; paying for a Vista licence was the ransom.

I tried to reinstall XP from its CD while preserving the data on the partition, but I got some frightening error messages in the process and aborted before risking the whole partition.

This could have been a serious crisis, but for two things -
  • I do all my real work on the Linux partition, and didn't really have anything to preserve from the Windows side. I just wanted to take a look around to verify that.
  • The Windows NTFS partition was perfectly accessible from Ubuntu, so I copied what I needed that way - didn't even need a thumb drive, just dragged icons onto the Ubuntu desktop. Hooray! I hadn't realized that read-only access to NTFS was so foolproof from Linux.
But what if I hadn't been a Linux freak? This could have been a real sackcloth-and-ashes event.

[If it hadn't been a dual-boot machine, I think I could have accomplished the same thing with an Ubuntu install CD, booting from it and copying the files to a USB drive.]

I don't think MS planned it this way. They just failed to plan a soft landing for their software's trial period ending. I'll remember that next time I play with a demo.

Wednesday, February 07, 2007

Microsoft Expression Web

Let me say two things in praise of Microsoft today:

1. They sponsored CodeMash.

2. I really, really like their new HTML editor, Expression Web. I like the split-screen, with the cursor that simultaneously shows each element in the WYSIWYG and HTML panes; I like the ease of generating in-line styles and moving them easily and cleanly to a separate CSS, I *love* the clean, uncluttered HTML it generates (*so* unlike FrontPage, ptui)

And, to my amazement, it's not Microsofty at all. It doesn't generate IIS-dependent code, you can preview the rendering in Firefox, etc. as easily as in IE. It's... respectful!

I'm officially asking my boss to buy us a copy. The one problem is... I prefer to work on Linux. Yes, I have a dual-boot machine, but going back and forth is a pain. So - can anybody recommend their favorite Linux-compatible HTML editors? Has anybody else tried Expression Web? Can you speak to how its features compare? Honestly, I've pretty much stuck with a text editor all this time, and I don't know if Expression Web's features are unique or routine.