Iterables and Iterators

by: George El., March 2019, Reading time: 4 minutes

In this post I will explain the what is an iterable and what is the difference between Iterators and Iterables. An iterable is every object that can be iterated. In order for this to happen it must implement the __iter__ method, and the __iter__ method must return an iterator object. An iterator object is an object that implements the __next__ method. This is how python loops through an iterable, it calls the next method until it gets an exception. It is possible the object to implement both methods and thus be both an iterable and iterator. In fact all iterators are also iterables.

Lets see a list first and use the dir function to see if it contains the __iter__ method

>>> l = [1, 2, 3, 4]
>>> dir(l)
['__add__', '__class__', '__contains__', '__delattr__', 
'__delitem__', '__dir__', '__doc__', '__eq__', '__format__', 
'__ge__', '__getattribute__', '__getitem__', '__gt__', 
'__hash__', '__iadd__', '__imul__', '__init__', '__init_subclass__', 
'__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', 
'__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', 
'__rmul__', '__setattr__', '__setitem__', '__sizeof__', '__str__', 
'__subclasshook__', 'append', 'clear', 'copy', 'count', 'extend', 
'index', 'insert', 'pop', 'remove', 'reverse', 'sort']
>>>

As we can see above the list has the __iter__ method but not the __next__ method so it is an iterable and not an iterator. Lets call this method. When we call __iter__ we get back a list iterator

>>> l.__iter__()
<list_iterator object at 0x0000023921BE2278>

Lets store this iterator in a variable called list_iterator. Now lets see if this object implements the __next__ method

>>> list_iterator= iter(l)
>>> type(list_iterator)
<class 'list_iterator'>
>>> dir(list_iterator)
['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', 
'__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', 
'__init__', '__init_subclass__', '__iter__', '__le__', '__length_hint__', 
'__lt__', '__ne__', '__new__', '__next__', '__reduce__', '__reduce_ex__', 
'__repr__', '__setattr__', '__setstate__', '__sizeof__', '__str__', 
'__subclasshook__']
>>>

indeed it implements the __next__ method. Now if we call this method repeatedly we will get the elements of the list

>>> list_iterator.__next__()
1
>>> list_iterator.__next__()
2
>>> list_iterator.__next__()
3
>>> list_iterator.__next__()
4
>>> list_iterator.__next__()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

We notice that after we exhausted the values, we get a StopIteration exception. Now as I said python Python has two built in methods iter() and next() that they call iter and next respectively, so we can do this instead

>>>l = [1,2,3,4]
>>> li = iter(l)
>>> next(li)
1
>>> next(li)
2
>>> next(li)
3
>>> next(li)
4
>>> next(li)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

So lets see another example

>>> l = [1, 2, 3]
>>> z = zip(l,'abc')
>>> z
<zip object at 0x0000023921BFA048>

Zip return a zip object. Lets see if zip is an iterable.

>>> '__iter__' in dir(z)
True
>>> '__next__' in dir(z)
True

Zip implements both iter and next so it is an iterator and iterable. We can call next method to get the values.

>>> next(z)
(1, 'a')
>>> next(z)
(2, 'b')
>>> next(z)
(3, 'c')
>>> next(z)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

Once an iterator is exhausted I cannot reloop, I need to get another iterator. Lets see with an example

>>> z = zip(l,'abc')
>>> list(z)
[(1, 'a'), (2, 'b'), (3, 'c')]
>>> list(z)
[]

the second time I call list(z) I get an empty list because the iterator has reached the end. This is an important difference between iterators and iterables. Another thing to note is that calling iter on an iterator always returns itself, so if I do iter(z) is z I get True while iter(l) is l I get false

>>> z = zip([1,2,3,4],'abcd')
>>> z
<zip object at 0x000001F3C625CEC8>
>>> iter(z) is z
True
>>> l = [1,2,3,4]
>>> iter(l) is l
False

zip, enumerate, open(‘file’), map return iterator, so you can iterate over it only once. range, dictionary.keys(), .values(), .items() return iterable so you can iterate over it many times. Lets see an example.

>>> f = open("test.txt")
>>> next(f)
'1\n'
>>> next(f)
'2\n'
>>> next(f)
'3\n'
>>> next(f)
'4'
>>> next(f)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

open returns an object which is an iterator. You can iterate over it only once. If I try to iterate over it again I will get nothing.

>>> f = open("test.txt")
>>> for line in f:
...     print(line,end='')
...
1
2
3
4
>>> for line in f:
...     print(line)
...

>>>

another way to do it

>>> f = open("test.txt")
>>> list(f)
['1\n', '2\n', '3\n', '4\n']
>>> list(f)
[]
>>>

On the contrary range always return an iterable. So I can do:

>>> list(range(10))
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> list(range(10))
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
comments powered by Disqus