Try Monique.io for free and save 93% of the time needed to set up dashboards and alerts.
Small parts of Python builtins functionality you might overlook
I've programmed in Python for years, but I'm still encountering usage of Python builtins - strings, dicts and lists - in a way I didn't know is possible, or in a way that simplifies some common tasks. This is a list of such random findings:
  • startswith and endswith methods of string/unicode accept tuples of arguments. Instead of:
    name.startswith('a') or name.startswith('b')
    do this:
    name.startswith(('a', 'b'))
  • if you want to delete specific charactes from a string, you can use translate method with None as a first argument and a string containing all characters to delete as a second:
    >>> 'aaXbbYbbX'.translate(None, 'XY')
    'aabbbb'
    
    It does not work with unicode/Python 3 strings.
  • dict's get method is very handy. d.get(k) returns None if k is not a key of d . Instead of:
    if 'key_name' not in d:
        raise Exception()
    else:
        val = d['key_name']
    
    I prefer to write:
    val = d.get('key_name')
    if val is None:
        raise Exception()
    
    (of course if None can be a valid value in a dictionary it will not work).
  • new string formatting style is handy for implementing __str__ / __repr__ :
    def __str__(self):
        return 'id={self.id}, name={self.name}, score={self.score}'.format(self=self)
    
  • s.split(sep, maxsplit) will split the string at most maxsplit times. If you need only the first chunk, you can do:
    chunk, rest = s.split(sep, 1)
    
  • I'm not sure it's a documented API, but you can call set 's methods as static methods and give a list of sets as arguments:
    >>> set.union({1}, {2})
    set([1, 2])
    >>> set.union(*[ {i % 2 for i in range(10)} ])
    set([0, 1])
    
  • set 's methods isdisjoint , issubset , issuperset are helpful to avoid loops and write more concise code. For example, to check if vals contains invalid values, instead of:
    for v in vals:
        if v in invalid_vals:
            raise Exception('Invalid val')
    
    you can do
    if not set(invalid_vals).isdisjoint(set(vals)):
        raise Exception('Invalid val')
    
  • similarly, one can sometimes construct sets from lists and avoid writing nested loops using intersection or union operations. If you want to iterate over keys present in two dictionaries, instead of:
    for k in d1:
        if k in d2:
            found(k)
    
    you can do:
    for k in set(d1.keys()) & set(d2.keys()):
        found(k)
    
15 comment(s)
Anonymous, 2013-04-22 20:13:32
the last example can be written more concisely as
for k in itertools.chain(d1, d2):
    found(k)
Author, 2013-04-22 20:23:02
But this way you will possibly call found(k) multiple times.
Anonymous, 2013-04-22 20:41:31
oh intersection my bad
Ben Hoyt, 2013-04-22 23:50:56
One I learned about just the other day was the two-argument form of iter, iter(func, sentinal): http://docs.python.org/2/library/functions.html#iter -- the iteration will call func() till it returns the sentinal value.
Matt Vallee, 2013-04-23 03:42:34
Your example of dict.get(key) accepts an optional parameter after the key if you don't want it to return None. For example:

>>> a = {'key': 'value'}
>>> b = a.get('fake_key','xyz')
>>> print b
'xyz'
bunny, 2013-04-23 04:52:11
For string formatting, you can also do

def __str__(self):
    return 'id={id}, name={name}, score={score}'.format(**vars(self))
Anonymous, 2013-04-23 05:05:38
what is the benefit of set.union() instead of '|' ? e.g. set.union({1}, {2}) vs. {1} | {2}

also, instead of:
    if not invalid_vals.isdisjoint(vals):
        raise Exception('Invalid val')

why not:
    if vals & invalid_vals:
        raise Exception('Invalid val')
Author, 2013-04-23 06:26:05
@Anonymous
Well using set.union as if it was a static method is more natural if you already have a list of sets.
Isdisjoint for sure can be faster and probably better describes intent.

@bunny
I didn't know about the vars functions, in this case it seems it just returns self.__dict__.
Santiago, 2013-04-23 14:25:22
Great post. I love to learn all this new stuff that are useful in the day to day.
wdahab, 2013-04-23 14:31:45
>>> set.union(*[ {i % 2 for i in range(10)} ])
set([0, 1])

unnecessary to dereference the list, as set comprehensions work.
>>> {i%2 for i in xrange(10)}
set([0, 1])

perhaps you meant to have the list be a list of sets and the close curly bracket is misplaced? in which case should be:
>>> set.union(*[ {i%2} for i in range(10)])
set([0, 1])
Peter Russell, 2013-04-25 11:24:51
In the last example, you can just do set(d1) rather than set(d1.keys()) (assuming d1 and d2 are dicts), since dicts are iterables of their keys.

In fact doing that is better, as IIRC there's a fast path in the interpreter for converting a dict into a set, which avoids re-hashing the key values.

Otherwise, a very nice article. The first two points were new to me, and will be useful.
Anonymous, 2013-04-25 15:28:01
if 'key_name' not in d:
    raise Exception()
else:
    val = d['key_name']

could better be written:

try:
    val = d['key_name']
except KeyError:
    raise CustomException()
Author, 2013-04-25 19:45:52
@Peter Russell
I just noticed that even faster would be using viewkeys() and avoiding creation of sets. View objects support union and intersection.

@Anonymous, 2013-04-25 15:28:01
It depends on what part of other code this snippet is. If you don't want to raise Exception() but treat is as a normal situation and execute some logic it would look strange to do it inside "except" block. But again it is more a matter of style than a hard rule.
Sebastian Dahlgren, 2013-04-26 13:37:14
Thanks for some interesting examples. However, if you don't need rest, then change

chunk, rest = s.split(sep, 1)

to

chunk, _ = s.split(sep, 1)

Cheers!
Tim, 2013-05-03 11:05:19
Sebastion, I don't using '_' as a variable name is good practice. It's usually used for i18n to denote a translatable string, i.e. _("Hello")