Me

Ian Rolfe's Journal.

Random Jibberings on Programming

This may be the reason why The Men That Will Not Be Blamed For Nothing...
Me
[info]ianrolfe
...cannot be killed by conventional weapons.

This album was released yesterday by "The Men That Will Not Be Blamed For Nothing" and, perhaps surprisingly, is even better than their debut "That's what I call Steampunk, Volume 1" or "The Steampunk Album That Cannot Be Named For Legal Reasons" as it is now called since the wallies at EMI got in a huff over nothing.

Buy it.
  • Leave a comment
  • Add to Memories

"'NoneType' object has no attribute 'tags'"
Me
[info]ianrolfe
Well I've struggled with this one for a while; my django app (indeed the whole project) was just giving 500 (internal Server) errors. The traceback wasn't especially informative, it looked as if it was having trouble rendering the DebugError template (which should have pointed me in the right direction much sooner!).

Immediately prior to the problem i'd done an SVN update across a number of apps on the server all of which share code so I had no idea what had updated and caused the problem, it could have been any one of a few hundred files.

But eventually by placing debugs in appropriate places it turned out that the problem was in a context manager in which someone had added a call to django.template.add_to_builtins naming a module that I hadn't updated into my working copy. As a result, the templating module failed to import it, and when it tried to output the debug page it also failed to import it silently, resulting in 'None' being added to the list of built-ins, hence the final error!

Since Django uses the templating system to generate error pages, I suppose it's inevitable that if there are circumstances where errors in custom template filters/tags might disable the templating mechanism, resulting in nasty 500 errors rather than a nice error page with a useful traceback.

Oh well you learn something every day!

Mysql: Wildcard patterns on mysqldump
Me
[info]ianrolfe
The mysqldump utility for MySQL doesn't allow you to dump just the tables that match an expression. You can either dump the whole database, or a list of named tables. Since in Django all the tables for a given application are prefixed with the application name, it would be useful to do this. I did this on unix with the command:
mysqldump my_database $(mysql my_database -B --column-names=False 
              -e "show tables like 'myapp_%'" | xargs ) >dumpfile.sql
(nb: This is all one line, linebreaks put in for clarity)
This produces the tables "myapp_*" in the dump file. Note the use of the SQL wildcard '%' in the actual command; the inner SQL command "show tables like..." is where we get the list of tables to dump.

The '-B' option forces the output to be in "batch" mode, i.e. plain columns seperated with tabs. Otherwise MySQL will draw nice little borders and columns with dashes and plus signs. The "--column-names=False" stops it adding a spurious extra line with the column name (which in this case is just "Tables matching..." or something like that).

On Windows? Hmm well I chickened out and just dumped from a unix client on the one occasion I had to partially dump a MySQL database on a windows server, but if thats not for you I guess you'd have to:
  • Write a batch file or script in VB or something to do it,
  • Install Cygwin and pretend you're on a unix machine,
  • or something else? Leave your suggestion in the comments!!
Tags: ,

Vim and the home & end keys.
Me
[info]ianrolfe
I've been playing with Lubuntu in a VM running on my work PC (Yes, its actually work). Now in the vi-emacs wars I'm kind of on the side of vi, partly because its the first unix editor I learned (and am too lazy to learn another) but mainly because it's the one your most likely to find on a.n.other unix box.

All is fine with Lubuntu, vi (well, actually vim) is installed & worked except pressing the home and end keys just inserted lines with "H" and "F" respectively.

Lots of people seem to be having this problem, but there were no answers that worked for me.

However, Since these keys work fine in the shell and everywhere else, I decided it was the key mapping in vim; so I created some new mappings (by googling around, it's not something I've had to do before) in my ~/.vimrc and all was well
# For some reason home and end keys are not mapping properly.
# Home key
imap <esc>OH <esc>0i
cmap <esc>OH <home>
nmap <esc>OH 0
# End key
nmap <esc>OF $
imap <esc>OF <esc>$a
cmap <esc>OF <end>
Note: The first mapping is esc-capital-o-h -> esc-zero-i; same kind of thing with the third one.

Validating Email address: Python
Me
[info]ianrolfe
Well I googled around to find a python function to validate email addresses for my Django 1.1 application (In 1.2 there is a email validation routine in django.core.validators) and this was the result of merging various strategies I found (with special regard to this activestate post)
import re
:
:
def valid_email(email):
    if email==None:
        return False
    return re.match(r"^[a-zA-Z0-9._%-+]+\@[a-zA-Z0-9._%-]+\.[a-zA-Z]{2,}$", email)!=None
Works a charm ;)

Python: Reversing strings.
Me
[info]ianrolfe
For something that I was working on, I wanted to replace only the LAST occurance of a character in a string with another one. The replace() method allows you to restrict the number of replaced characters to 1, but it always replaces the FIRST occurance. So it seemed the simplest way is to reverse the string, replace the occurance and reverse it back again.
Since strings hold so many similarities to lists, I was surprised to find that it doesn't have a 'reverse' method like lists do. The 'reversed()' iterator works, but unless your stepping through your string backwards, its not much use. However, a bit of googling and headscratching came up with the simple solution!:
>>> s="This is a string example"
>>> s[::-1]
'elpmaxe gnirts a si sihT'
>>> s.replace('i','o',1)
'Thos is a string example'
>>> s[::-1].replace('i','o',1)[::-1]
'This is a strong example'
>>>
The [::-1] slice operation is an arcane idiom, but nothing that judicious use of comments can't sort out for those that will be maintaining your code later!

While true: Compact idiom or idiotic idea?
Me
[info]ianrolfe
For some purposes where a loop has to end under some fairly complex conditions I like to do something like this:
while True:
    [do something]
    if testvar=2 or name='Slartibartfast':
        break
    [perhaps do some more]
Now, to me this seems a perfectly good idea, it provides an easy to understand loop with the potential for multiple exit conditions, without requiring the use of any otherwise pointless boolean variables:
loop_now = True
while loop_now:
   [do something]
   if testvar=2 or name='Slartibartfast':
       loop_now = False
   if loop_now:
       [perhaps do some more]
Which seems to me
  • Introduces a semantically pointless boolean variable
  • makes the code unnecessarily complex.
However, some of my colleagues seem to feel that my version is somehow morally lacking in that the use of the break statement like this is tantamount to using a goto. I'd argue that the boolean is effectively a goto anyhow!
What would you say?
Tags:

Python, environment variables and the os module...
Me
[info]ianrolfe
I know it says it in the documentation, but if you have to call a module that expects an environment variable to be set, and want to set it in your script at runtime rather than reconfigure your server/PC, don't use os.putenv(), just assign to the os.environment variable:
os.putenv('DJANGO_SETTINGS_MODULE','settings')         # NO!!
os.environment['DJANGO_SETTINGS_MODULE'] = 'settings'  # YES
The reason for this is that os.putenv() actually puts the values specified into the process environment (and that variable will therefore be seen by any subprocesses you spawn), but does not put it into the os.environment variable which is initialised bu python from the environment variables when the interpreter starts. However, setting the variable directly in the os.environment dictionary does a os.putenv() call in the background - which is kind of the opposite way around from what you might expect.
Calling os.putenv() will not "simulate" the value of an environment variable because the vast majority of code looks in the os.environment dictionary when it needs a value, rather than calling os.getenv().
Tags:

Pickling
Me
[info]ianrolfe
Just a quickie here, I guess its kind of obvious, but you can put multiple objects in a data file and extract them as seperate objects again. Just use the same file handle in multiple pickle.dump() calls to write the objects. When pickle.load() is called, then it will stop at the end of the object, leaving the file pointer where it needs to be for a subsequent call:
# Test pickling of multiple objects...
import pickle

class One(object):
    def __init__(self,name):
        self.name=name

    def __repr__(self):
        return "One('%s')" % self.name

class Two(object):
    def __init__(self,name):
        self.name=name

    def __repr__(self):
        return "Two('%s')" % self.name

one = One("This is one")
two = Two("This is two")

f = open("pickle.dat","wb")
pickle.dump(one,f,-1)
pickle.dump(two,f,-1)
f.close()

print one, two
del one, two

f = open("pickle.dat","rb")
anotherone = pickle.load(f)
anothertwo = pickle.load(f)
f.close()

print anotherone, anothertwo

Python idioms #1: de-duplicating lists
Me
[info]ianrolfe
It's easy really,
>>> a=[4,2,6,5,3,6,4,2]
>>>
>>> a
[4, 2, 6, 5, 3, 6, 4, 2]
>>> list(set(a))
[2, 3, 4, 5, 6]
>>>

It works with lists of non-numerical values too:
>>> a=[(0,0),(2,3),(0,0),(3,2)]
>>> a
[(0, 0), (2, 3), (0, 0), (3, 2)]
>>> list(set(a))
[(3, 2), (0, 0), (2, 3)]

It doesn't work so well with classes:
>>> class Test(object):
...     def __init__(self, v):
...             self.value = v
...     def __repr__(self):
...             return "Test(%s)" % (self.value,)
...
>>> Test(20)
Test(20)
>>> a=[Test(2), Test(4), Test(5), Test(2), Test(3)]
>>> a
[Test(2), Test(4), Test(5), Test(2), Test(3)]
>>> list(set(a))
[Test(2), Test(4), Test(3), Test(5), Test(2)]
>>>

Creating the set doesn't remove the "duplicate" values, because it tests by 'id', which will be different even if all the attributes are the same. Defining a __eq__() or __cmp__() function doesn't seem to make a difference, either.

You are viewing [info]ianrolfe's journal