Using Python for Serious Development

Some time ago an old friend asked a question on the Python-IL mailing list, wondering what we thought of using Python for serious development. I think my response back then was pretty interesting, so I'm reproducing it here (edited cosmetically only). Today I would've probably put it differently, but the essence would be the same.

From: Ori Peleg
Mon, 16 Apr 2007 23:17:55 -0700

Yes, you should program in Python! (or a similar agile
language) :-)

My main points:

 - Python is excellent for refactoring (beats C++ and, yes,
   Java too)
 - Don't move an inch with unit and system tests
 - These tests are much easier to write in Python
 - Compiler-checked interfaces aren't that important
 - After a few serious successes, Python is a very serious
   contender in my company.

Now to our application's story:
We ported a large application core to Python (original was in
C++ - we wrote it too).
Our motivation was maintainability - we were sure we could
reduce that hulking core to a smaller and simpler creature.
This really did happen, to the point that the project never
went into maintenance mode - it suddenly became useful for
things we didn't predict :)

We got performance benefits as well (yes, we were surprised).
The reasons were straightforward - we had many more options
when refactoring the code, and separated a lot of logic
neatly (for example extracted DB caching to a decorator -
"AOP" in a language that needs a new term :-)

The gains we ended up getting could have been achieved in
C++, obviously, but they only became apparent after
refactoring with Python.

We also wrote a quick sample client/server application with
Twisted, which was later modified in 2 weeks by someone
different and transformed into an excellent production
application, that's still working flawlessly after 6 months
with no maintenance. All this with about 300 lines of simple,
practically stupid, "I understand this code the first time I
see it" code (actually, this is probably the reason for the
quality).

Now, every time someone says "client/server" people think
about Python and Twisted, and even a large legacy network
client is being replaced with a small Twisted client soon.

How come Python is so amazing for refactoring?
First, you need really good tests. I don't miss the compiler
when I have really good tests.
The good side is that to develop at good quality and speed
you need really good tests in any language, so you're not
losing anything.  These tests are also so much easier to
write in Python - we get more and better tests, and spend
less time writing them. (Whatever language you use for the
system, you should probably test in a language like Python).

Once these tests are in place, refactoring in Python with a
good text editor and some basic tools is so much smoother
than C++ or Java, even with Eclipse's really cool 'Refactor'
menu.
Those actions that are trivial in Eclipse ( e.g. 'Rename
Method') are a little more complicated in Python, but not by
much (remember the good tests).
On the other hand, anything that isn't in Eclipse's context
menu is absolutely horrible to do in Java, to the point that
I've met many good Java developers that don't think anything
else exists at all :) In Python, I've found it to be a snap.
And when you add all the more and less dynamic tools that can
be put to good use, I'm one happy refactorer. Decorators,
bound methods, generator expressions, and the like give me
many ways to make the code simpler that aren't nearly as
accessible in Java (or barely possible in C++).

Why aren't interfaces that important? The reason I like
interfaces is as documentation that the compiler happens to
check. It's convenient, but I don't terribly miss it in the
presence of (a) good documentation, and (b), you guessed it,
good tests :-D
I don't have much experience with zope.interface, so I don't
know how easy it is to use or how much of the benefit from
interfaces you get.
I must say about this point that I'm comparing a full
static-typing approach (e.g. Java) to a full dynamic-typing
approach (e.g. Python) - give me dynamic typing any day. A
hybrid statically- and dynamically-typed language like Groovy
could work better, I don't have the experience to say.

orip.