Week 11 - Error Handling and Debugging¶

There is no programming without error handling, as your programs will inevitably encounter errors. In fact, we've been dealing with errors since we started to learn Python back in week 3. Every error is different, some errors will only happen in certain conditions, some errors will take hours to solve, if not days. Here's a very fun error to read :-)

Nevertheless, it's a programmer's duty to handle errors responsibly. First, let's go over the different types of errors.

Syntax errors¶

Every language has rules to make correct sentences, and these rules are called syntax. Naturally, every programming language has their own rules of making valid "sentences", and so it also has syntax to follow.

Here are some examples (from the slides) that break the rules, and therefore result in "syntax error":

In [1]:
# Not adding a ":" at the end of a for loop:
for i in range(10)
     print(i)

  Cell In[1], line 2
    for i in range(10)
                      ^
SyntaxError: expected ':'
In [2]:
x = 10.0
# Forgetting closing ) of an pair of parenthesis:
a = ((x+5)*12+4

print(a)

  Cell In[2], line 3
    a = ((x+5)*12+4
        ^
SyntaxError: '(' was never closed

Note that the invalid syntax isn't print(a) above, but it's caused by the interpreter trying to find the missing parenthesis in the above statement.

In [3]:
# Mistaking assignment (=) for equality (==) in a boolean statement
while x = 4:
    s += x

  Cell In[3], line 2
    while x = 4:
          ^
SyntaxError: invalid syntax. Maybe you meant '==' or ':=' instead of '='?

These are generally the easiest errors to solve.

Type errors¶

Type errors are caused by using the wrong type, or using mismatched types together. Here are some examples of type errors:

In [4]:
astr = 'hello'
print(astr ** 3)  # No power operator defined for strings

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[4], line 2
      1 astr = 'hello'
----> 2 print(astr ** 3)  # No power operator defined for strings

TypeError: unsupported operand type(s) for ** or pow(): 'str' and 'int'
In [5]:
x = 5.0

# Can't index a float type
print(x[0])

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[5], line 4
      1 x = 5.0
      2 
      3 # Can't index a float type
----> 4 print(x[0])

TypeError: 'float' object is not subscriptable
In [6]:
# Can't add a dictionary to a string
{'hello': 'world'} + 'goodbye'

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[6], line 2
      1 # Can't add a dictionary to a string
----> 2 {'hello': 'world'} + 'goodbye'

TypeError: unsupported operand type(s) for +: 'dict' and 'str'
In [7]:
# Adding (or concatenating) a string and a number
'66' + 33

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[7], line 2
      1 # Adding (or concatenating) a string and a number
----> 2 '66' + 33

TypeError: can only concatenate str (not "int") to str

Run-time Errors¶

Run-time errors are those errors that are only encountered when running your code. These have correct syntax, and use the correct types. They require a bit of closer inspection to understand where the error is coming from.

The count() example below finds the count of numbers between 1 and 999 (inclusive) that are divisible by a given number n.

In [8]:
def divisible(a, b):
    return a % b == 0

def count(n):
    s = 0
    for i in range(1, 1000):
        if divisible(i, n):
            s += 1
    return s
In [9]:
count(10)
Out[9]:
99
In [10]:
count(2)
Out[10]:
499

As we see, count() generally works for the numbers that we provide. However, nothing stops us from using 0 as the input number:

In [11]:
count(0)

---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
Cell In[11], line 1
----> 1 count(0)

Cell In[8], line 7, in count(n)
      4 def count(n):
      5     s = 0
      6     for i in range(1, 1000):
----> 7         if divisible(i, n):
      8             s += 1
      9     return s

Cell In[8], line 2, in divisible(a, b)
      1 def divisible(a, b):
----> 2     return a % b == 0

ZeroDivisionError: integer modulo by zero

This is a run-time error that we see when we try to divide a number by 0, which is what happens when we call count(0).

With list-like structures, we frequently see indexing errors such as the one below:

In [12]:
a = 'hello'
print(a[4])
print(a[5])

o

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Cell In[12], line 3
      1 a = 'hello'
      2 print(a[4])
----> 3 print(a[5])

IndexError: string index out of range

In OOP, when we try to access a function or class variable that does not exist, we encounter an attribute error:

In [13]:
class Text:
    pass
t = Text()
t.parse()

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[13], line 4
      1 class Text:
      2     pass
      3 t = Text()
----> 4 t.parse()

AttributeError: 'Text' object has no attribute 'parse'

There are many different types of run-time errors, one can inspect the table in the chapter about run-time errors for more examples.

Logical errors¶

Finally, logical errors are when our code does not behave as we expect. These errors are sometimes the hardest ones to solve -- they might not even terminate our program!

Here's a naive error to make when we forget operator precedence. Suppose we want to code

$$ y = \frac{x}{x+1} $$

like this:

In [14]:
x = 4
y = x / x + 1
print(y)

2.0

We expected the result to be $y = \frac{4}{5} = 0.8$, so why is this the case?

This is because / takes higher precedence than +, so this will be read by Python as y = (x / x) + 1, which is always 2 (unless x = 0, in which case we'll have a problem).

To fix this, we should add parenthesis ourselves to make the grouping more explicit:

In [15]:
y = x / (x + 1)
print(y)

0.8

Here's another example from function scoping.

Let's try to change the value of x, while we're in a():

In [16]:
x = 10

def a():
    x = 5  # Update value of x?
    print('x in a():', x)

a()
print('x outside of a():', x)

x in a(): 5
x outside of a(): 10

This would be a logical error since we're trying to change the value of x. We can solve this by specifying the global keyword:

In [17]:
x = 10

def a():
    global x
    x = 5  # Update value of x?
    print('x in a():', x)

a()
print('x outside of a():', x)

x in a(): 5
x outside of a(): 5

Sometimes we forget to return values when we're in a function, like this example:

In [18]:
def f(x, y):
    if x < y:
        return 10
In [19]:
f(5, 3) + f(10, 20)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[19], line 1
----> 1 f(5, 3) + f(10, 20)

TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

f(5, 3) has type NoneType because we don't return a value when x >= y inside f().

Logical errors are not limited to these examples, and can be caused by anything and everything. That's what makes them harder to solve.

Dealing with Errors¶

Now that we know how what types of errors exist, we can try our best to eliminate them using a checklist-type of approach.

First of all, just being mindful of good coding practices goes a long way. It's easier to solve a problem if you don't introduce it in the first place, right? When we're writing code, we should think of the general outline of the code, as well as some cases where our code may fail (such as with specific inputs).

In particular, if we're dealing with numbers for example, we may try to detect if the input that we received can be handled by our code. This is called input validation, and it's a good way to eliminate errors:

In [20]:
import math

def root(n):
    return math.sqrt(n)

Taking the square root of a negative number will result in a run-time error:

In [21]:
root(-5)

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[21], line 1
----> 1 root(-5)

Cell In[20], line 4, in root(n)
      3 def root(n):
----> 4     return math.sqrt(n)

ValueError: math domain error

So we can eliminate this possibility by checking that a given input is positive:

In [22]:
import math

def root(n):
    if n < 0:
        print("Can't take the square root of a negative number!")
        return None
    return math.sqrt(n)
In [23]:
root(-5)

Can't take the square root of a negative number!

In the example above, in the case of invalid input, we return None to show that the result is not valid. Sometimes we can return special error codes (such as -1 here, as no real number could have a square root of -1) or other tokens.

In general, to signal that we've encountered an unexpected situation, one can use exceptions. Exceptions are special objects that signify errors and are handled differently than other things that we've seen until now. In fact, the run-time errors (ValueError, etc.) that we've seen are actually exceptions.

For example, let's use exceptions above:

In [24]:
import math

def root(n):
    if n < 0:
        raise ValueError("Can't take the square root of a negative number!")
    return math.sqrt(n)
In [25]:
root(5)
Out[25]:
2.23606797749979
In [26]:
root(-5)

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[26], line 1
----> 1 root(-5)

Cell In[24], line 5, in root(n)
      3 def root(n):
      4     if n < 0:
----> 5         raise ValueError("Can't take the square root of a negative number!")
      6     return math.sqrt(n)

ValueError: Can't take the square root of a negative number!

First of all, we don't return exceptions, we raise (or "throw") them. This way, you can see your call stack (the traceback above) and specifically which line your code had an error.

You can also provide a message to be shown along with the error, like we did above.

$\tiny{\text{Trivia: You technically can return exceptions (Python won't complain), but it's the same as returning any normal object, and you lose out on the special exception handling utilities.}}$

Since you don't return exceptions, you don't have to be in any function to raise an exception:

In [27]:
raise ZeroDivisionError('hello')

---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
Cell In[27], line 1
----> 1 raise ZeroDivisionError('hello')

ZeroDivisionError: hello

Finally, an exception will "bubble" up in your program until it is handled, or the program terminates. Here's an example of an exception interrupting a function while it is executing:

In [28]:
def f():
    print('hello from f')
    raise ValueError('except!')
    print('goodbye from f')

def g():
    print('hello from g')
    f()
    print('goodbye from g')
    
g()

hello from g
hello from f

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[28], line 11
      7     print('hello from g')
      8     f()
      9     print('goodbye from g')
     10 
---> 11 g()

Cell In[28], line 8, in g()
      6 def g():
      7     print('hello from g')
----> 8     f()
      9     print('goodbye from g')

Cell In[28], line 3, in f()
      1 def f():
      2     print('hello from f')
----> 3     raise ValueError('except!')
      4     print('goodbye from f')

ValueError: except!

Finally, you can even create your own exception type, by inheriting from the Exception class.

Here's the example from class, where we wrote a "safer" fibonacci number function that only accepts positive integers as its input.

In [29]:
class FibonacciError(Exception):
    # Don't need to do anything
    pass
In [30]:
def fibonacci(n):
    if type(n) != int:
        # raise ValueError()
        raise FibonacciError('Fibonacci only defined for natural numbers')
    if n < 0:
        # raise ValueError('Error: n cannot be negative')
        raise FibonacciError('Error: n cannot be negative')
    if n == 0 or n == 1:
        return 1
    else:
        return fibonacci(n-1) + fibonacci(n-2)
In [31]:
fibonacci(10)
Out[31]:
89
In [32]:
fibonacci(-5)

---------------------------------------------------------------------------
FibonacciError                            Traceback (most recent call last)
Cell In[32], line 1
----> 1 fibonacci(-5)

Cell In[30], line 7, in fibonacci(n)
      3         # raise ValueError()
      4         raise FibonacciError('Fibonacci only defined for natural numbers')
      5     if n < 0:
      6         # raise ValueError('Error: n cannot be negative')
----> 7         raise FibonacciError('Error: n cannot be negative')
      8     if n == 0 or n == 1:
      9         return 1
     10     else:

FibonacciError: Error: n cannot be negative
In [33]:
fibonacci(1.5)

---------------------------------------------------------------------------
FibonacciError                            Traceback (most recent call last)
Cell In[33], line 1
----> 1 fibonacci(1.5)

Cell In[30], line 4, in fibonacci(n)
      1 def fibonacci(n):
      2     if type(n) != int:
      3         # raise ValueError()
----> 4         raise FibonacciError('Fibonacci only defined for natural numbers')
      5     if n < 0:
      6         # raise ValueError('Error: n cannot be negative')
      7         raise FibonacciError('Error: n cannot be negative')

FibonacciError: Fibonacci only defined for natural numbers
In [34]:
fibonacci('hello')

---------------------------------------------------------------------------
FibonacciError                            Traceback (most recent call last)
Cell In[34], line 1
----> 1 fibonacci('hello')

Cell In[30], line 4, in fibonacci(n)
      1 def fibonacci(n):
      2     if type(n) != int:
      3         # raise ValueError()
----> 4         raise FibonacciError('Fibonacci only defined for natural numbers')
      5     if n < 0:
      6         # raise ValueError('Error: n cannot be negative')
      7         raise FibonacciError('Error: n cannot be negative')

FibonacciError: Fibonacci only defined for natural numbers

Now, just as we raise an exceptions, we must also deal with thrown exceptions.

We deal with exceptions using try...except blocks, like below:

In [35]:
value = 3.5

try:
    result = fibonacci(3.5)
except FibonacciError as err:
    print('Encountered an error:', err)
    

Encountered an error: Fibonacci only defined for natural numbers

There's no traceback, because we caught the exception with the try-except block. We can check for multiple types of exceptions by chaining except statements:

In [36]:
a = [1, 2, 3]

try:
    # Try these out to get different errors
    print(a[4])  # invalid indexing 
    # print(a[0] / 0)  # zero division
    # print(a ** 2)  # type error -- caught by the final except
except ZeroDivisionError:
    print('Divide by zero')
except ValueError:
    print('Some other kind of value error')
except IndexError:
    print('Container indexing error!')
except: # Catches any exception
    print('Some other error')

Container indexing error!

Of course, it goes without saying that if you are not the one responsible to handle an exception, you should not catch it so that it is dealt with accordingly (by some other code).

Writing test cases¶

It's a good practice to test your implementation of a specific feature of your code so that it works well with expected situations. That way, when the code changes (maybe months later), you can be sure that it works on some known cases.

Curious students can check out the unittest module for writing test cases in a more modular manner.

In [37]:
# Calculates y for a given x, a, b, c in ax**2 + bx + c = y equation 
def parabola(x, a, b, c): 
    return a * (x ** 2) + b * x + c

# Some example test cases for a given parabola
a = 2
b = -4
c = 5
test1 = parabola(0, a, b, c) == 5  # Passes through (0, 5)
test2 = parabola(-2, a, b, c) == 21  # Passes through (-2, 21)
test3 = parabola(3, a, b, c) == 11  # Passes through (3, 11)

print('Passes test 1:', test1)
print('Passes test 2:', test2)
print('Passes test 3:', test3)

if test1 and test2 and test3:
    print('All tests passed!')

Passes test 1: True
Passes test 2: True
Passes test 3: True
All tests passed!

Debugging¶

Debugging is the act of removing bugs (errors) from programs (why "bug"? see here for historical context).

There are multiple ways of debugging, and the easiest method is to add print statements to show the values of variables between execution.

We solved the problem of Arimu and Hats from the workbook in class, and in the coding process we printed our results along the way:

In [38]:
def buy_hats(l, num):
    # Should sort the list first 
    l = sorted(l)
    print(l)
    count = 0
    i = 0
    while num >= 0: # num: amount of money left
        print('Remaining money:', num)
        if i >= len(l):
            print('No items left to buy')
            break
        if l[i] <= num:
            num = num - l[i]
            print('Bought', l[i], ' | Remaining money:', num)
            count += 1
            i += 1
        else:
            print("Can't buy anything")
            break
    return count
In [39]:
buy_hats([12, 3, 7, 5, 4, 8], 12)

[3, 4, 5, 7, 8, 12]
Remaining money: 12
Bought 3  | Remaining money: 9
Remaining money: 9
Bought 4  | Remaining money: 5
Remaining money: 5
Bought 5  | Remaining money: 0
Remaining money: 0
Can't buy anything
Out[39]:
3
In [40]:
buy_hats([4, 3, 6, 2, 5, 5], 27)

[2, 3, 4, 5, 5, 6]
Remaining money: 27
Bought 2  | Remaining money: 25
Remaining money: 25
Bought 3  | Remaining money: 22
Remaining money: 22
Bought 4  | Remaining money: 18
Remaining money: 18
Bought 5  | Remaining money: 13
Remaining money: 13
Bought 5  | Remaining money: 8
Remaining money: 8
Bought 6  | Remaining money: 2
Remaining money: 2
No items left to buy
Out[40]:
6

For more complicated debugging, it's better to use something called a debugger, which is a special program that can run while your executing your code and gives you finer control over code execution. IDEs such as PyCharm, VSCode, etc. all have nice debugging screens where you can see values of variables, function scopes, and more; but the simplest application of using a debugger is by using the pdb module available with Python.

You can use pdb by importing it. Afterwards, to stop your code in a specific location, you can add a breakpoint by using pdb.set_trace().

You can use the following commands to inspect your code:

  • print(var) will print the value of var in the debugger. (shortcut: p var)
  • next will execute the current line, and move on to the next line (shortcut: n)
  • If the current line calls a function, step will go inside of the function call. (shortcut: s)
  • list will print the surrounding lines in the code (shortcut: l)
  • where will show the stack trace (the stacked calls of functions) (shortcuts: w, bt)
  • help shows help. (shortcut: h)

There are more commands (continue, etc.) to use, it's very fun to play with pdb.

In [41]:
import pdb

# Fixed version of the sum_and_delete from the slides
def sum_and_delete(l):
    total = 0
    for _ in range(len(l)):
        # Uncomment the line below, and play with the debugger!
        # pdb.set_trace()
        total += l[0]
        del l[0]
        
    return total
In [42]:
sum_and_delete([1, 2, 3, 4, 5])
Out[42]:
15