Seven Intermediate-Level Tips and Tricks for Python Lists

The list is everywhere in Python. Up your game with these tips and tricks.

This article is for Python programmers who have gone beyond the basics of using lists, but would like to see some interesting concepts. It isn’t very advanced and should be easy for you to understand.

1. Ensure that a List Contains Unique Elements.

Sets contain unique elements but are unordered. On the other hand, lists can contain duplicate elements but retain their insertion order.

This example shows one way to get the benefits of both sets and lists — an ordered sequence of unique elements.

We are subclassing the Python built-in collections.UserList object. We could subclass list, or use collections.abc to create a new abstract base class of type MutableSequence. But those two approaches entail significantly more work to correctly implement all of the __dunder__ methods.

Here, we’re only interested in ensuring that sequences passed to the constructor, or any new elements appended to the end of the list, are unique.

Line #39 is doing all the work. If the count() method is false: greater than 1, then skip the append.

2. Find all the Index Values of a Matching a Test Condition

What do you do if you want to efficiently test a large list to find the indices of all matching elements? This can be quite easy to get wrong: either by being too slow or consuming too much memory.

This is one way, to use an iterator and a generator.

Note that I’m using doctests here instead of assert statements. Line #13 coalesces all the iterator’s yielded results into a list.

It is also possible to use the get_indices() method in a for loop:

for k in get_indices("The jolly green giant", "e"):
    print(k)

3. Flatten a List of Lists into one Super List

What do we do if we want to flatten a really long list, perhaps many thousands of items, and each one is itself a list? Again, I’ve opted to use the technique of not using return, and instead using yield.

The function flatten_nested_lists doesn’t really complete in a traditional sense. Every time a value is computed it is yielded for the calling programming to make use of. This is a really important trick, also known as “lazy evaluation”. If my code decided I only needed the first 10 values, I can stop there and not need to calculate everything.

The chain.from_iterable() method is very useful, and it does what the name implies! According to the docs, this method is roughly equivalent to:

def from_iterable(iterables):
    # chain.from_iterable(['ABC', 'DEF']) --> A B C D E F
    for it in iterables:
        for element in it:
            yield element

“Roughly Equivalent” in Python-speak means, more or less: this code is horribly inefficient but returns the same result, in almost all cases.

4. Implement a FrozenList

Python doesn’t contain a FrozenList object out of the box. The suggestion was rejected by the Python community, not least because it’s easy for us to implement ourselves.

I’ve implemented a subclass of the collections.UserList object, and overriden the methods which allow changes to the list’s elements.

Lines 5–8 define a simple decorator, which raises an exception on whatever methods it decorates.

5. Create an Autoappend List

If we want to append or overwrite items to an evergrowing list, we need to choose to use append() at the right time. Otherwise, we’ll get an error.

The following class helps its users by automatically appending a new element to the list if you’re trying to write to the n+1 index element.

This example creates a list of the character’s ordinal positions in the ASCII table. Normally we would be required to use the append() method on line 29, but the class takes care of this case on lines 20–21.

6. Create a List Which Only Accepts Specific Object Types

Of course, it is possible to create a List which only accepts certain object types or only certain values, similar in effect (but not syntax) to this C# code, using generics:

List<Part> parts = new List<Part>();

Python uses Duck typing, but also has excellent support for type checking when using MyPy, and Python in very recent versions. See this article #7 on how to install and use MyPy to statically analyze your usage of types in Python.

Again, subclassing UserList is an easy way to accomplish this: simply by selecting all the methods which accept input and validating it contains what we want.

In normal usage, the class URLFilteredList and the method recursive_key_values would be located in separate code files and imported here.

We’d be left with the code in lines 47–53 which is super simple. We’re appending every value in a dictionary to the list, but the only values which survive past the filter are well-formatted URLs.

7. Use Map and Reduce on Lists

Clearly, the Python community thinks that functional-style programming is inferior. Map, reduce, filter and anonymous functions are widely considered to be an obfuscated way to write Python, and that comprehensions are the preferred method.

You will see Python code which uses the functional programming paradigm, and here is an example performing a simple calculation using three parameters x, y and z. The values for these parameters are stored in the lists a, b, and c. This is perfectly acceptable Python, even if it takes some practice to be able to read it.

On line 10 I should have not used a list of literal values, but instead taken the list called answers as the input to the reduce function. I felt it was better exposition to use the literal values :)

Hopefully you enjoyed this article! Look out for the next one — I’ll be talking about tuples. If you liked this article, you should check out the one I wrote about strings:

This is a companion article to the introduction:

Leave your comments if you noticed things to improve!