TurboGears and Oracle on OTN

Once, using TurboGears with Oracle was the kind of kooky idea you could only read about in the blog posts of obscure crackpots.

No more! The current top story at Oracle Technology Network is Daniel Rubio's article on using TurboGears with Oracle. Read, use, and enjoy! It's also got a permanent home at the OTN Scripting Language Technology Center.

Thanks to Oracle for all the great press lately, and to Daniel and Przemek for writing it!

PyCon: 141 submissions!

We weren't very good about getting word out for the PyCon call for proposals, and as the deadline approached, there weren't a lot of submissions in the queue. We worried about whether we'd messed up.

Oh, we of little faith! The last couple days before the deadline, the floodgates opened. We got 141 talks submitted! (And about 40 tutorials!)

And they're great submissions! If I had my way, I'd attend about 3/4 of them personally. Unfortunately, for a 3-day conference, we'll be lucky if we can fit half of them into the schedule. Well, I did suggest running PyCon on a 24-hour basis, but for some reason that wasn't accepted.

So now we've got to somehow narrow down the field to what we can fit in the con, and that's a painful process. Quite a few excellent talks won't go on the schedule simply because there's no room.

The good news is that we've got lots of space set aside for multiple tracks of Open Space sessions, which don't require any approval. So, in essence, we may get two PyCons running side-by-side - a traditional one with scheduled talks, and an unconference-like bunch of Open Space activity.

Too much good stuff... I'm definitely going to have to spawn some processes to see all I want to see.

New Python articles at OTN

There's a new series of articles on Oracle & Python over at the OTN Scripting Language Technology Center. Przemyslaw Piotrowski presents a detailed investigation of cx_Oracle, covering very nicely some aspects I'd glossed over in all my introductory articles. Great job, Przemek!

Microformat slides

Anybody who saw my Microformats talk, whether at the Dayton Web Standards Meetup or the DMA Linux SIG, can see my slides here or get a tarball here.

If you weren't at my talk, and are looking for a microformats presentation that will stand on its own, you're better off looking through the talk archives at the Microformats wiki. My talk isn't very self-explanatory without me standing there to talk about it.

CodeMash 2008

CodeMash is a great, multi-language agile programming conference here in Ohio. I had a great time last year, and next year looks even better.

CodeMash v2.0.0.8

Jan. 9 - 11, 2008

Kalahari Waterpark Resort

7000 Kalahari Drive

Sandusky, OH 44870USA

877-525-2427

The speaker committee has announced two sets of accepted abstracts for CodeMash 2008, with more still to come. Already, there's a lot for a Python fan to be excited about!

=============
Bruce Eckel: "Why I Love Python"
=============
Bruce Eckel has given hundreds of presentations throughout the world, published over 150 articles in numerous magazines, was a founding member of the ANSI/ISO C++ committee and speaks regularly at conferences. He provides public and private seminars & design consulting in OO Design, Python, Java and C++.

============
Leah Culver: "Getting Started with Django"
============
Leah Culver founded Pownce with her friends Kevin Rose and Daniel Burka as a way of sending messages, links, files and events to each other. Leah is the lead developer for the site, which has become one of the largest sites using the Django framework. Leah loves the challenge of developing a web application from scratch and writes about her experiences as a software developer at leahculver.com.

============
Kevin Dangoor: "Overview of the Dojo JavaScript Toolkit"
============
Kevin is the product manager at SitePen and the founder of the TurboGears open source web application framework. He has held positions in software development, management and sales engineering. He has previously spoken at CodeMash, PyCon, EuroPython and GLSEC and is the co-author of the book "Rapid Web Applications with TurboGears".

Kevin's talk isn't directly about Python, but I expect he'll cover how to tie Dojo into Turbogears. And if he doesn't offer it freely, I'm going to grill him on it!

Finally, my talk was accepted, too! It's basically a repeat of my Ohio LinuxFest "Introduction to Python" talk that used vPython to simulate a solar system, but since the audience this time will be a group of high-powered programmers, I should be able to amp it up and cover more ground. I'll try to dovetail with the more advanced talks at CodeMash so that (for instance) you'll be able to understand Leah's Django talk even if you arrived at CodeMash without knowing any Python.

There are also talks on Domain-Specific Languages, Silverlight (which, via IronPython, looks like a great way to Python-power webpages on the browser side)... and the speaker committee isn't even finished selecting talks yet.

Can. Not. Wait! Between this and PyCon in Chicago, it's truly an Ohio Pythonista's dream come true.

Python at MIT

There was a bit of buzz about a year ago when MIT announced plans to experiment with an alternative version of their electrical engineering/computer science (Course 6) curriculum - one that would start students with a Python-based course instead of the famous Scheme-based 6.001. It was exciting news, but I hadn't heard anything about it since then.

Today, a look through the department's webpage shows that the experiment has come quickly to fruition, and the new curriculum is the standard for this year's freshmen.

6.01, Introduction to EECS I, with Python (and robots! Sounds FUN!)

I'm dying to know more about this! I'm going to hit up school friends for info, but maybe somebody here has more details to supply?

I still meet plenty of neckties who have never heard of any language besides C/Java/.NET (hey, it's the Midwest, and it's the Air Force), don't know why Python matters, and aren't interested in understanding its merits directly - only want to know "who's using it?". "MIT teaches it to all their EE/CS students" is a pretty good one-liner response. I'm not above leveraging MIT's name recognition. I did the time, after all (as a chemical engineer, not EE/CS, but still). Granted, it never won world domination for Scheme, but I bet it is the source of a very large fraction of the fame that Scheme does have.

SQL*Plus alternatives (at InOUG)

I spoke last Thursday at the Indiana Oracle Users Group on "The Command Line Lives: SQL*Plus and Alternatives".

I still need to get the full, S5 format presentation posted somewhere; for now, you can see the raw text of the presentation here.

It's an interesting topic, and one I'd like to get drawn more deeply into. Some nice things have been done in producing open-source command-line alternatives to SQL*Plus, including

• gqlplus
• Senora
• YASQL
• sqlpython

Unfortunately, the only one whose development currently seems to be active is sqlpython, and there's just me and Luca Canali working on that. I've just added functionality to sqlpython to let it fail over to sessions of the other tools (when "set failover true" is issued; doesn't work on Windows). That way, the exciting features of Senora and YASQL can be used from within sqlpython, instead of trying to remember which tool has which feature. It's quirky, though, since a full-fledged independent session of each tool is actually maintained "under the covers".

I'm dreaming of expanding sqlpython to the point where it incorporates all Senora and YASQL's features natively. Tragically, my dreams cannot yet be downloaded from the Cheese Shop.

xkcd

I promise not to make a habit of posting about xkcd.

But only because I'm counting on you to read it daily yourself.



I printed this today and then wandered the office, looking for someone to show it to, gradually coming to the horrifying realization that I don't work with anyone who would get it.

cx_Oracle and Oracle XE on Ubuntu

When I first posted about using Oracle XE on Ubuntu, I thought that getting cx_Oracle working with XE was pretty easy. I was wrong! Last night, I installed on a "clean" machine, and it was a lot harder; perhaps last time, I benefited from packages I'd already installed, meeting secret prereqs without knowing it.

[EDIT: As of June 2008, a streamlined set of instructions is here.]

Fortunately, I did work out the necessary steps.

1. sudo apt-get install libc6-dev
   
   If you don't, you'll get

   In file included from /usr/lib/gcc/i486-linux-gnu/4.1.2/include/syslimits.h:7,
   from /usr/lib/gcc/i486-linux-gnu/4.1.2/include/limits.h:11,
   from /usr/include/python2.5/Python.h:18,
   from cx_Oracle.c:6:
   /usr/lib/gcc/i486-linux-gnu/4.1.2/include/limits.h:122:61: error: limits.h: No such file or directory
   In file included from cx_Oracle.c:6:
   /usr/include/python2.5/Python.h:32:19: error: stdio.h: No such file or directory

   ... when you try to install cx_Oracle.
   
2. 
   export ORACLE_HOME=/usr/lib/oracle/xe/app/oracle/product/10.2.0/server/
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
export PATH=$ORACLE_HOME/bin:$PATH
   
   If you don't, you'll get

   error: cannot locate an Oracle software installation

   or

   
   ImportError:libclntsh.so.10.1:cannot open shared object file: No such file or directory

3. Download the source code tarball, unzip and unpack, cd into it, and
   sudo python setup.py install
   You will get

   error in cx_Oracle setup command: Oracle home does not refer to an 8i, 9i, 10g or 11g installation

   because setup.py doesn't account for XE. You need to hack it.
   
   Download the patch useXE.diff and run
   patch setup.py useXE.diff
   
   Now run
   sudo python setup.py install
   again.
   
   I'm going to submit a patch to the cx_Oracle folks, so hopefully it will be fixed for the next release.
   
   [EDIT: Anthony Tuininga, cx_Oracle's developer, has taken my patch, improved it, and incorporated it into the cx_Oracle trunk; as of the next cx_Oracle release beyond 4.3.3, this is fixed. This sort of thing is what makes open source fun!
   
   To grab the patched version from the cx_Oracle development trunk right now, install Subversion on your machine, then
   svn co https://cx-oracle.svn.sourceforge.net/svnroot/cx-oracle/trunk cx-oracle
   ]
   
   
4. If you're accessing Oracle XE on your own machine, start your listener. On my machine, at least, the XE installation didn't do that.
   sudo su - oracle
export ORACLE_HOME=/usr/lib/oracle/xe/app/oracle/product/10.2.0/server/
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
export PATH=$ORACLE_HOME/bin:$PATH
lsnrctl startexit
   
   You may want to edit your .bashrc to do this; otherwise, you'll need to for every session in which you access Oracle.
   
   If you're using Oracle XE as a client to access Oracle (XE or otherwise) on another machine, skip this step, but you'll need to make sure the SID you're looking for is in $ORACLE_HOME/network/admin/tnsnames.ora.
   
   
5. At last, you can start using cx_Oracle! See the DB-API2 Cheat Sheet for some quick syntax reminders.

EDIT: comment from teambob

I mistakenly deleted the following comment:

I installed via easy_install, but there was a bit of a trick.

After setting the environment variables I installed using the following command: sudo -E easy_install cx_oracle

The "-E" option to sudo causes all your environment variables to be passed to easy_install

Ohio LinuxFest slides & code

Thanks to everyone for your attendance and feedback at my OLF talk! I had a great time!

A tarball with everything you need to duplicate my presentation is here.

It's pretty complex, so it includes a README file. (I used s5 slides generated by rst2s5, a Socrates file from Bruce, and eight directories of code files. Don't worry, though, it's very lean - just 60K.)

I'm speaking on Python at OLF

What's this? You haven't registered for LinuxFest yet?

But I'm (absurdly honored to be) one of the speakers!

Introduction to Python and/or Solar System Demolition
An introduction to Python programming. Demonstrates the language by developing a graphical solar system gravitation simulation in real-time.
September 29th, 2007, 3:50 - 4:40 PM

Greater Columbus Convention Center

400 North High Street

Columbus, OH 43215 USA

And I'm bringing my own projector this time, just in case... (plus I've got nVidia drivers installed, which should help...)

Ohio LinuxFest: register now!

Drop everything and register for




It's free - but the registration deadline is Sep. 14 - this Friday! Move, move, move!

Ohio LinuxFest
September 29th, 2007

Greater Columbus Convention Center

400 North High Street

Columbus, OH 43215 USA

Microformats talk

Have I mentioned that the Dayton Web Standards Meetup is a great group? It's brimming over with good ideas for every sort of web writer. Very highly recommended!

This coming Thursday, I'll be speaking on Microformats! Microformats are a wonderful way to make your information available to automated tools, while mandating nothing about what else you do on your page. They're great fun, they're catching on, and I'm looking forward to it!

5:30, Sep. 6: Microformats talk at Dayton Web Standards Meetup
at Panera Bread on 1203 Brown Street. Dayton OH 45409 USA (937) 223 6700 microformats web meetup See you there!

Okay, I admit, I wrote this post partially to practice writing a microformat. Install Operator into your Firefox. Now, with one click (each), you can grab a vCard for Panera, get a map there, or pop the event inot your calendar program. Oooooh!

Dayton-Oracle is for real

The Dayton-Oracle mailing list I set up a couple years ago is bearing fruit. The Dayton Oracle User Group is real! We live! We had an organizational pre-meeting last night with five great people who were willing to take up the burden of making this happen.

We had a lot of rich discussion about the way we want to approach running the group. In many ways, we're going to break from the traditional format and content of Oracle User Groups and adopt some practices better-known from the open-source world: more interactive, user-driven content; zero-cost bring-your-own-food arrangements instead of catered lunches with meeting fees and sponsored content; evening rather than daylong meetings; etc. I think these are good practices on their own merits. Also, there are already good traditional user groups in Columbus, Cincinnati, and Indianapolis, so if we want a successful niche between them, it makes sense for us to differentiate; we're not likely to be a better OOUG than OOUG, after all. I hope Dayton people who do already attend OOUG or GCOUG will continue to do so, and will enrich themselves yet further with our meetings.

Watch this space for further announcements, including our first meeting time (sometime early October) and a link to our super-awesome website (once it's built).

openSUSE and the bleeding edge

I fear I'm about to sound both boastful and ungrateful... but my boss refreshed my work laptop with this monster I have dubbed "DellZilla". Cool, huh? Definitely - except that you need a pack mule to actually move it. Once upon a time, laptops could actually fit in your lap.

The other problem is that I haven't found a Linux distribution that comes with the right drivers for it. Ubuntu Feisty was hopeless - all incarnations (u-, ku-, and xu-) were flummoxed by the screen and failed to start xwindows, or gave an utterly black empty screen.

Next I tried openSUSE, and was in some ways pleasantly surprised. It copes OK with whatever video voodoo is required for this monster screen. It automatically noticed that I had a Windows Vista partition and included it as a boot option. In fact, it includes "Restart and boot directly into Windows" as a menu option, so you can go get a cola while switching OS's instead of hovering over the keyboard waiting to catch the boot loader. Considerate! And it comes with all the cutting-edge goodies like Beagle. (And freeciv! Eek!)

One significant complaint is that, if you don't have wireless connectivity while you're doing the openSUSE installation, YAST won't even record the URLs of the repositories you'll need for software. When you need to download the good stuff later on, you'll need to hunt around for the URLs (psst, they're here) and add them to your list of sources by hand. For a SuSE newbie, that's a pain.

But I got through that. The real problem is that I've got no support for the mac80211 wireless card. Trying to get a driver installed for that has been an adventure in frustration. There's a driver here, but it appears to my amateurish eye as though YAST installs the Linux kernel-source into locations where the mac80211 makefile absolutely does not expect it, and trying to hack them into agreement is a step too far for me. It's like being back in the Bad Old Pre-Ubuntu days when Linux was something I only wished I could use.

So here I am, typing on a zillion-dollar laptop and tethered to the wall like it's 1994, and trying to figure out my next move. This says Sabayon 3.4 supports mac80211. I hadn't even heard of Sabayon before this week, and I've read good things about it, but I wonder if it's just too fancy. It installs from DVDs; I've never even tried burning one of those. And I can't even get its blasted homepage to come up under Firefox or Konqueror. But whatever; I need wireless. If you don't hear from me for a while, it's because I need to compile my own drivers for the keyboard, or maybe the AC adapter.

birthday list

Pardon my long absence from the blogosphere. One cause has been that my workplace blocked access to Blogger, all Blogger-hosted blogs, all mailing lists anywhere, etc. I'm becoming thoroughly frustrated. The message it seems to send is, "We don't desire the fruit of your work as much as we value our infantilizing mistrust of your professionalism. We pay for the services of skilled adults yet believe that we got delinquent children instead."

But anyway. That's not why I'm posting. I've been fairly nomadic lately, migrating my work from machine to machine and experimenting with various operating systems, and it's made me crave some good organizational software. Since my birthday is coming up, I thought I would ask the Universe to produce it and see what happens.

1. I need help keeping track of the difference between files I've worked on - stuff I want to keep when I move or rebuild - and stuff I can leave behind because I just downloaded it from somewhere and I'll just download it afresh on a new system anyway. Then it'll be easy to back up and transfer my work. This software will note when I modify a file with a text editor, IDE, image editor, OpenOffice, or whatnot, and will flag it as my own personal work - a "keeper". Stuff that comes from downloads, from expanding zipfiles, automated processes and loggers, etc. will all be flagged "ephemeral".

2. I need help reorganizing my folder structure. I'd like something that can scan my drive and notice that my photos are scattered in four different places, short text notes are all over, I've got four copies of the same file in four directories under three names, etc., and propose and facilitate a consolidation plan.

3. A nice tag-based file access system That could actually replace #2 and be a key component of #1.

If you know of this stuff already existing, speak up! If you want to write it, hooray! If you want to wait several years until I have the skill and the time to do it myself, well... keep reading my blog, I guess. If you are saying, "You wouldn't need any of this if you exercised organizational discipline when creating your files", then please note that I am Sticking My Tongue Out At You.

get a job, sha na na na

If you're reading this and you're interested in switching jobs, let me know...

... because I'm constantly being asked these days whether I know people looking for work. It's a good feeling, but it would be a better feeling if I had somebody to refer them to. And I'm sure I'm not the only one getting these inquiries, so if you've felt stuck writing boringCode for a pointy-haired boss, it's a great time to let your friends know.

This applies to Python people as well as Oracle. That's a neat change; when I got involved in Python a few years ago, the assumption was that most Python programmers had day jobs using more boring languages, and could only use Python in their passion-driven midnight coding sprees. Now, employer interest is very high - it seems employer interest has actually grown faster and overtaken the population of Python coders. That's especially noteworthy when you consider how quickly someone can become a Python programmer.

Anyway, good times for Py and Ora geeks. Woo hoo!

Kubuntu

For Feisty, just to learn something new, I thought I'd try Kubuntu (KDE Ubuntu) instead of Ubuntu this time around.

And... and... I think I'm a believer.

For what I've done so far, the extra information KDE provides is very well thought-out, just what I need to know when I need to know it. For instance...

• When I want to remove a USB drive or somesuch, but something is still using it, Ubuntu (like Windows, including Vista) only tells me that something is using it. Kubuntu tells me what is using it.
• Alt-Tabbing through my open windows gives more information than just the icon of each running program - enough so that I can distinguish between my eight different command prompt windows.
• Hovering over an icon pops up the file properties. Nice! I mean, why else would I be hovering over an icon, except that I was fumbling for the right-mouse button so I could find out more?
• Clicking my networking icon provides all the information I need to choose at a single glance, and then one more click connects me. Also, Kubuntu seems to jump to conclusions and do the right thing for networking - when there's only one wireless network around, Kubuntu just connects me without waiting to be asked.

The downsides I've found so far...

• I really liked Ctrl-+ to increase the font size of GNOME windows. That's important when trying to read laptop screens in adverse environments (like outside). I haven't found a KDE equivalent.
• The built-in text editor, Kate, bungles the spaces/tabs in Python files. That's one good reason to install Komodo (free editor or the real thing) immediately.

Real gurus can discuss the relative strengths of KDE and Gnome for hours, of course, and I thank God that I've never been caught in such a debate. I'm just saying that KDE is making a really good first impression on me, and that's after years of Gnome experience.

Extensible CheckBox List

Introducing a brand-new widget for TurboGears: Extensible CheckBox List!

To use it:

1. easy_install xcbl
2. In your controller.py,
   from xcbl.widgets import ExtensibleCheckBoxList
3. In your form's WidgetsList,
   pets = ExtensibleCheckBoxList(options=['cats','dogs','turtles'])
   (exactly the same as you would for an ordinary CheckBox List widget)

The result - a CheckBox List with an "other" field that lets the user hand-enter new options - as many as they want.

Unfortunately, the new items will appear beneath the "Other:" box, not above it, unless you upgrade your MochiKit to 1.4. Details on how to do that and more are here.

Developing it was lots of fun - it really shows how easily a relative amateur can contribute to TurboGears. I hope to write up a "Developing and Contributing TurboGears Widgets for Dummies" document soon, based on the experience.

cleansing table and column names

What kind of joker thought "BASE POC'S" would make a good column name? Sheesh, MS Access. (Not to mention apostrophe abuse. See Bob's Quick Guide to the Apostrophe, You Idiots.)

To my surprise, I googled "script rename invalid columns oracle" and got nothing useful. So here's a chance to play with Oracle regular expressions.

You may be thinking, "If they're illegal column names, how can they get in the database in the first place?" With "quotation marks", that's how. In fact, you can

CREATE TABLE "!!!My Rockin' Table!!!"
  ( "*^@(@  @!!!" VARCHAR2(30) 
                  CONSTRAINT " -- d00d --" PRIMARY KEY,
    "miXiNG CASE is TEH KEWList" VARCHAR2(30) ); 

and, not only will all your hideous names be used verbatim, but they will be case-sensitive. If you take advantage of this feature, you should be slapped. I guess the reason Oracle makes this possible in the first place is so that an import can be accepted from a source - like MS Access - regardless of how awful its names are.

Anyway, here's a PL/SQL cure for bad table and column names.
(link to code)

/*
Renames tables and columns to be Oracle-legal.

SET LINESIZE 200 before running to avoid unwanted page breaks.
Run from an account with privileges to
  - access v$reserved words 
  - alter the appropriate tables

Thanks to Eddie Awad for his description of v$reserved words:
http://awads.net/wp/2007/01/10/what-does-reserved-y-really-mean/

Catherine Devlin, catherinedevlin.blogspot.com, 11 May 2007, v0.1
*/
CREATE OR REPLACE PROCEDURE cleanse_names IS
  new_obj_name VARCHAR2(30);
  CURSOR tbl_names IS
    SELECT owner,
           table_name
    FROM   all_tables
    WHERE  dropped = 'NO';
  CURSOR col_names IS
    SELECT owner,
           table_name,
           column_name
    FROM   all_tab_columns
    JOIN   all_tables USING (table_name, owner)
    WHERE  dropped = 'NO';

  PROCEDURE do(this IN VARCHAR2) IS
  BEGIN
    DBMS_OUTPUT.PUT_LINE(this || ';');
    -- Uncomment if you're feeling decisive
    -- EXECUTE IMMEDIATE(this);
  END do;
  FUNCTION legalize(name IN VARCHAR2, replacement_letter IN VARCHAR2)
  RETURN varchar2
  IS
    new_name VARCHAR2(30);
    is_reserved NUMBER(11,0);
  BEGIN    
    -- must begin with a letter
    new_name := REGEXP_REPLACE(name,'^[^[:alpha:]]',replacement_letter);
    -- and contain only letters, numbers, #,_,$
    new_name := REGEXP_REPLACE(new_name,'[^[:alnum:]#_$]','_');
    -- and mixing case is evil
    new_name := UPPER(new_name);
    -- and we must respect Oracle reserved words
    SELECT count(*)
    INTO   is_reserved
    FROM   v$reserved_words
    WHERE  keyword=new_name
    AND    (reserved='Y' OR res_semi='Y');
    IF is_reserved > 0
    THEN
      new_name := SUBSTR(new_name, 1, 29) || '_';
    END IF;
    RETURN new_name;
  END legalize;
BEGIN
    DBMS_OUTPUT.PUT_LINE('set scan off');
    FOR col IN col_names
    LOOP
      new_obj_name := legalize(col.column_name, 'C');
      IF new_obj_name != col.column_name
      THEN
        do('ALTER TABLE ' || col.owner ||
           '."' || col.table_name ||
           '" RENAME COLUMN "' || col.column_name ||
           '" TO ' || new_obj_name);
      END IF;
    END LOOP;
    FOR tbl IN tbl_names
    LOOP
      new_obj_name := legalize(tbl.table_name, 'T');
      IF new_obj_name != tbl.table_name
      THEN
        do('ALTER TABLE ' || tbl.owner ||
           '."' || tbl.table_name ||
           '" RENAME TO ' || new_obj_name);
      END IF;
    END LOOP;
END cleanse_names;