Testing, Debugging, and Version Control
It is important to note that, in programming, the actual task of writing code is not the only element of the process. There are other administrative procedures that play important roles in the pipeline that are often overlooked. In this section, we will discuss each task one by one and consider the process of implementing them in Python, starting with testing.
Testing
In order to make sure that a piece of software that we have written works as we intended and produces correct results, it is necessary to put it through specific tests. In software development, there are numerous types of testing that we can apply to a program: integration testing, regression testing, system testing, and so on. One of the most common is unit testing, which is our topic of discussion in this section.
Unit testing denotes the focus on inpidual small units of the software, as opposed to the entire program. Unit testing is typically the first step of a testing pipeline—once we are reasonably confident that the inpidual components of our program are working correctly, we can move on to test how these components work together and see whether they produce the results we want (with integration or system testing).
Unit testing in Python can be easily implemented using the unittest module. Taking an object-oriented approach, unittest allows us to design tests for our programs as Python classes, making the process more modular. Such a class needs to inherit from the TestCase class from unittest, and inpidual tests are to be implemented in separate functions, as follows:
import unittest
class SampleTest(unittest.TestCase):
def test_equal(self):
self.assertEqual(2 ** 3 - 1, 7)
self.assertEqual('Hello, world!', 'Hello, ' + 'world!')
def test_true(self):
self.assertTrue(2 ** 3 < 3 ** 2)
for x in range(10):
self.assertTrue(- x ** 2 <= 0)
In the SampleTest class, we placed two test cases where we want to check whether two given quantities are equal or not using the assertEqual() method in the test_equal() function. Here, we test whether 23 - 1 is indeed equal to 7, and whether string concatenation in Python is correct.
Similarly, the assertTrue() methods used in the test_true() function test for whether the given parameter is evaluated True or not. Here, we test whether 23 is less than 32, and whether the negative of perfect squares of integers between 0 and 10 are non-positive.
To run the tests we have implemented, we can use the following statement:
>>> unittest.main()
test_equal (__main__.SampleTest) ... ok
test_true (__main__.SampleTest) ... ok
----------------------------------------------------------------------
Ran 2 tests in 0.001s
OK
The produced output tells us that both of our tests returned positive. One important side note to keep in mind is that if you are running a unit test in a Jupyter notebook, the last statement needs to be as follows:
unittest.main(argv=[''], verbosity=2, exit=False)
As a result of the fact that the unit tests are to be implemented as functions in a Python class, the unittest module also offers two convenient methods, setUp() and tearDown(), which are to be run automatically before and after each test, respectively. We will see an example of this in our next exercise. For now, we will move on and talk about debugging.
Debugging
The term debugging literally means the removal of one or many bugs from a given computer program, thus making it work correctly. In most cases, a debugging process follows a failed test where it is determined that there is a bug in our program. Then, to debug the program, we need to identify the source of the error that caused the test to fail and attempt to fix the code related to that error.
There are multiple forms of debugging that a program might employ. These include the following:
- Print debugging: Arguably one of the most common and elementary methods of debugging, print debugging involves identifying the variables that might play a role in causing the bug, placing print statements for those variables at various places in our program so that we can track the changes in the values of those variables. Once a change in the value of a variable is found to be undesirable or unwanted, we look at where specifically that print statement is in the program and therefore (roughly) identify the location of the bug.
- Logging: If instead of printing the values of our variables to standard output, we decide to write the output to a log file, this is called logging. Logging is often done to keep track of specific events taking place in the execution of the program we are debugging or simply monitoring.
- Tracing: To debug a program, we, in this case, will follow the low-level function calls and execution stack of the program when it executes. By looking at the order in which variables and functions are used from that low-level perspective, we can identify the source of the error as well. Tracing can be implemented in Python using the sys.settrace() method from the sys module.
In Python, it is quite easy to employ print debugging, as we simply need to use print statements. For more complex functionalities, we can use a debugger, a module/library that is specifically designed for debugging purposes. The most dominant debugger in Python is the built-in pdb module, which used to be run via the pdb.set_trace() method.
Starting from Python 3.7, we can opt for a simpler syntax by placing calls to the built-in breakpoint() function. At each place where a breakpoint() function is called, the execution of the program will pause and allow us to inspect the behavior and current characteristics of the program, including the values of its variables.
Specifically, once the execution of the program reaches a breakpoint() function, an input prompt will appear, where we can enter a pdb command. There are many commands that you can take advantage of that are included in the documentation of the module. Some notable commands are as follows:
- h: For help, which prints out the complete list of commands you can use.
- u/d: For up and down, respectively, which move the running frame count one level in a direction.
- s: For step, which executes the instruction that the program is currently at and pauses at the first possible place in the execution. This command is very useful in terms of observing the immediate effect of a line of code on the state of the program.
- n: For next, which executes the instruction that the program is currently at and only pauses at the next instruction in the current function and when the execution is returned. This command works somewhat similarly to s, though it skips through instructions at a much higher rate.
- r: For return, which continues the execution until the current function returns.
- c: For continue, which continues the execution until the next breakpoint() statement is reached.
- ll: For longlist, which prints out the source code for the current instruction.
- p [expression]: For print, which evaluates and prints out the value of the given expression
Overall, once the execution of a program is paused by a breakpoint() statement, we can utilize a combination of the preceding different commands to inspect the state of the program and identify a potential bug. We'll look at an example of this in the following exercise.
Exercise 1.08: Testing for Concurrency
In this exercise, we will consider a well-known bug in concurrency- or parallelism-related programs called a race condition. This will serve as a nice use case to try out our testing and debugging tools. Since the integration of pdb and other debugging tools is still underdeveloped in Jupyter Notebooks, we will be working with .py scripts in this exercise.
Perform the following steps to complete this exercise:
- The setup of our program (which is implemented in the following steps) is as follows. We have a class that implements a counter object that can be manipulated by multiple threads in parallel. The value of an instance of this counter object (stored in its value attribute, initialized to 0) is incremented every time its update() method is called. The counter also has a target that its value should be incremented to. When its run() method is called, multiple threads will be spawned. Each thread will call the update() method, thus incrementing its value attribute a number of times that is equal to the original target. In theory, the final value of the counter should be the same as the target, but we will see that this is not the case due to a race condition. Our goal is to apply pdb to track the changes of variables inside this program to analyze this race condition.
- Create a new .py script and enter the following code:
import threading
import sys; sys.setswitchinterval(10 ** -10)
class Counter:
def __init__(self, target):
self.value = 0
self.target = target
def update(self):
current_value = self.value
# breakpoint()
self.value = current_value + 1
def run(self):
threads = [threading.Thread(target=self.update) \
for _ in range(self.target)]
for t in threads:
t.start()
for t in threads:
t.join()
This code implements the Counter class that we discussed earlier. Note that there is a line of code that sets the switch interval of our system; we will discuss this later.
- With the hope that the value of a counter object should be incremented to its true target, we will test its performance with three different target values. In the same .py script, enter the following code to implement our unit tests:
import unittest
class TestCounter(unittest.TestCase):
def setUp(self):
self.small_params = 5
self.med_params = 5000
self.large_params = 10000
def test_small(self):
small_counter = Counter(self.small_params)
small_counter.run()
self.assertEqual(small_counter.value, \
self.small_params)
def test_med(self):
med_counter = Counter(self.med_params)
med_counter.run()
self.assertEqual(med_counter.value, \
self.med_params)
def test_large(self):
large_counter = Counter(self.large_params)
large_counter.run()
self.assertEqual(large_counter.value, \
self.large_params)
if __name__ == '__main__':
unittest.main()
Here, we can see that in each testing function, we initialize a new counter object, run it, and finally compare its value with the true target. The targets for the test cases are declared in the setUp() method, which, as we mentioned previously, is run before the tests are carried out:
Run this Python script:test_large (__main__.TestCounter) ... FAIL
test_med (__main__.TestCounter) ... FAIL
test_small (__main__.TestCounter) ... ok
====================================================================
FAIL: test_large (__main__.TestCounter)
--------------------------------------------------------------------
Traceback (most recent call last):
File "<ipython-input-57-4ed47b9310ba>", line 22, in test_large
self.assertEqual(large_counter.value, self.large_params)
AssertionError: 9996 != 10000
====================================================================
FAIL: test_med (__main__.TestCounter)
--------------------------------------------------------------------
Traceback (most recent call last):
File "<ipython-input-57-4ed47b9310ba>", line 17, in test_med
self.assertEqual(med_counter.value, self.med_params)
AssertionError: 4999 != 5000
--------------------------------------------------------------------
Ran 3 tests in 0.890s
FAILED (failures=2)
As you can see, the program failed at two tests: test_med (where the final value of the counter was only 4,999 instead of 5,000) and test_large (where the value was 9,996 instead of 10,000). It is possible that you might obtain a different output.
- Rerun this code cell multiple times to see that the result might vary.
- Now that we know there is a bug in our program, we will attempt to debug it. Reimplement our Counter class by placing a breakpoint() statement between the two instructions in the update() method, as shown in the following code, and rerun the code cell:
class Counter:
...
def update(self):
current_value = self.value
breakpoint()
self.value = current_value + 1
...
- In the main scope of our Python script, comment out the call to the unit tests. Instead, declare a new counter object and run the script using the Terminal:
sample_counter = Counter(10)
sample_counter.run()
Here, you will see a pdb prompt appear in the Terminal (you might need to press Enter first to make the debugger proceed):
Figure 1.4: pdb interface
- Input ll and hit Enter to see where in the program we are pausing:
(Pdb) ll
9 def update(self):
10 current_value = self.value
11 breakpoint()
12 -> self.value = current_value + 1
Here, the output indicates that we are currently pausing between the two instructions that increment the value of our counter inside the update() method.
- Hit Enter again to return to the pdb prompt and run the p self.value command:
(Pdb) p self.value
0
We can see that the current value of the counter is 0.
- Return to the prompt and enter the n command. After this, use the p self.value command again to inspect the value of the counter:
(Pdb) n
--Return--
> <ipython-input-61-066f5069e308>(12)update()->None
-> self.value = current_value + 1
(Pdb) p self.value
1
- We can see that the value has been incremented by 1. Repeat this process of alternating between n and p self.value to observe that the value stored in self.value is not updated as we proceed through the program. In other words, the value typically stays at 1. This is how the bug manifests itself in large values of the counter, as we have seen in our unit tests.
- Exit the debugger using Ctrl + C.
Note
To access the source code for this specific section, please refer to https://packt.live/2YPCZFJ.
This section does not currently have an online interactive example and will need to be run locally.
For those who are interested, the bug of our program stems from the fact that multiple threads can increment the value of the counter at roughly the same time, overwriting the changes made by one another. With a large number of threads (such as 5,000 or 10,000, which we have in our test cases), the probability of this event taking place becomes higher. This phenomenon, as we mentioned previously, is called a race condition, and it is one of the most common bugs in concurrent and parallel programs.
Aside from demonstrating some pdb commands, this exercise also illustrates the fact that it is necessary to design tests to cover different situations. While the program passed our small test with the target being 5, it failed with larger values of the target. In real life, we should have the tests for a program to simulate a wide range of possibilities, ensuring that the program still works as intended, even in edge cases.
And with that, let's move on to the last topic of this chapter, version control.
Version Control
In this section, we will briefly talk about the general theory behind version control and then discuss the process of implementing it with Git and GitHub, arguably the most popular version control systems in the industry. Version control is to a programming project what backing up data is to regular files. In essence, version control systems allow us to save our progress in a project separately from our local files so that we can come back to it later on, even if the local files are lost or damaged.
With the functionalities that current version control systems such as Git and GitHub provide, we can also do a lot more. For example, the branching and merging features from these systems offer their users a way to create multiple versions of a common project so that different directions can be explored; the branch that implements the most preferred direction will then be merged with the main branch in the end. Additionally, Git and GitHub allow work between users on their platform to be seamless, which is greatly appreciated in team projects.
To understand the available features that we can take advantage of with Git and GitHub, let's go through the following exercise.
Exercise 1.09: Version Control with Git and GitHub
This exercise will walk us through all of the steps necessary to get started with Git and GitHub. If you do not have any experience working with version control yet, this exercise will be beneficial to you.
Perform the following steps to complete this exercise:
- First, if you haven't already, register for a GitHub account by going to https://www.github.com/ and sign up. This will allow you to host the files that you want to version control on their cloud storage.
- Go to https://git-scm.com/downloads and download the Git client software for your system and install it. This Git client will be responsible for communicating with the GitHub server. You know if your Git client is successfully installed if you can run the git command in your Terminal:
$ git
usage: git [--version] [--help] [-C <path>] [-c <name>=<value>]
[--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
[-p | --paginate | -P | --no-pager] [--no-replace-objects] [--bare]
[--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
<command> [<args>]
Otherwise, your system might need to be rebooted for the installation to take full effect.
- Now, let's start the process of applying version control to a sample project. First, create a dummy folder and generate a Jupyter notebook and a text file named input.txt with the following content in it:
1,1,1
1,1,1
- In the first cell of the Jupyter notebook, write a function called add_elements() that takes in two lists of numbers and adds them up element-wise. The function should return a list that consists of the element-wise sums; you can assume that the two parameter lists are of the same length:
def add_elements(a, b):
result = []
for item_a, item_b in zip(a, b):
result.append(item_a + item_b)
return result
- In the next code cell, read in the input.txt file using a with statement and extract the last two lines of the file using the readlines() function and list indexing:
with open('input.txt', 'r') as f:
lines = f.readlines()
last_line1, last_line2 = lines[-2], lines[-1]
Note that in the open() function, the second argument, 'r', specifies that we are reading in the file, as opposed to writing to the file.
- In a new code cell, convert these two strings of text input into lists of numbers, first using the str.split() function with the ',' argument to isolate the inpidual numbers in each line, and then the map() and int() functions to apply the conversion to integers element-wise:
list1 = list(map(int, last_line1[: -1].split(',')))
list2 = list(map(int, last_line2[: -1].split(',')))
- In a new code cell, call add_elements() on list1 and list2. Write the returned list to the same input file in a new line in the same comma-separated values (CSV) format:
new_list = add_elements(list1, list2)
with open('input.txt', 'a') as f:
for i, item in enumerate(new_list):
f.write(str(item))
if i < len(new_list) - 1:
f.write(',')
else:
f.write('\n')
Here the 'a' argument specifies that we are writing to append a new line to the file, as opposed to reading and overwriting the file completely.
- Run the code cell and verify that the text file has been updated to the following:
1,1,1
1,1,1
2,2,2
- This is the current setup of our sample project so far: we have a text file and a Python script inside a folder; the script can alter the content of the text file when run. This setup is fairly common in real-life situations: you can have a data file that contains some information that you'd like to keep track of and a Python program that can read in that data and update it in some way (maybe through prespecified computation or adding new data that was collected externally).
Now, let's implement version control in this sample project.
- Go to your online GitHub account, click on the plus sign icon (+) in the top-right corner of the window, and choose the New repository option, as illustrated here:
Figure 1.5: Creating a new repository
Input a sample name for your new repository in the form and finalize the creation process. Copy the URL to this new repository to your clipboard as we will need it later.
This, as the name suggests, will create a new online repository that will host the code we want to version control.
- On your local computer, open your Terminal and navigate to the folder. Run the following command to initialize a local Git repository, which will be associated with our folder:
$ git init
- Still in the Terminal, run the following command to add everything in our project to Git and commit them:
git add .
git commit -m [any message with double quotes]
Instead of git add ., you can replace . with the names of the files that you want to register with Git. This option is helpful when you only want to register a file or two, as opposed to every file you have in a folder.
- Now, we need to link our local repository and the online repository that we have created. To do that, run the following command:
git remote add origin [URL to GitHub repository]
Note that "origin" is simply a conventional nickname for the URL.
- Finally, upload the locally registered files to the online repository by running the following command:
git push origin master
- Go to the website for the online repository to verify that the local files we created have indeed been uploaded to GitHub.
- On your local computer, run the script included in the Jupyter notebook and change the text file.
- Now, we would like to commit this change to the GitHub repository. In your Terminal, run the following commands again:
git add .
git commit
git push origin master
- Go to or refresh the GitHub website to verify that the change we made the second time has also been made on GitHub.
With this exercise, we have walked through a sample version control pipeline and seen some examples of how Git and GitHub can be used in this respect. We also saw a refresher on the process of reading and writing to files in Python using the with statement.
Note
To access the source code for this specific section, please refer to https://packt.live/2VDS0IS.
You can also run this example online at https://packt.live/3ijJ1pM.
This also concludes the last topic of the first chapter of this book. In the next section, we have provided an activity that will serve as a hands-on project that encapsulates the important topics and discussions we have gone through in this chapter.
Activity 1.01: Building a Sudoku Solver
Let's test what we have learned so far with a more complex problem: writing a program that can solve Sudoku puzzles. The program should be able to read in a CSV text file as input (which contains the initial puzzle) and output the complete solution to that puzzle.
This activity serves as a warmup consisting of multiple procedures that are common in scientific computing and data science projects, such as reading in data from external files and manipulating that information via an algorithm.
- Use the sudoku_input_2.txt file from the GitHub repository of this chapter as the input file for our program by copying it to the same location as the Jupyter notebook you will be creating in the next step (or create your own input file in the same format where empty cells are represented with zeros).
- In the first code cell of a new Jupyter notebook, create a Solver class that takes in the path to an input file. It should store the information read from the input file in a 9 x 9 2D list (a list of nine sublists, each of which contains the nine values of inpidual rows in the puzzle).
- Add a helper method that prints out the puzzle in a nice format, as follows:
-----------------------
0 0 3 | 0 2 0 | 6 0 0 |
9 0 0 | 3 0 5 | 0 0 1 |
0 0 1 | 8 0 6 | 4 0 0 |
-----------------------
0 0 8 | 1 0 2 | 9 0 0 |
7 0 0 | 0 0 0 | 0 0 8 |
0 0 6 | 7 0 8 | 2 0 0 |
-----------------------
0 0 2 | 6 0 9 | 5 0 0 |
8 0 0 | 2 0 3 | 0 0 9 |
0 0 5 | 0 1 0 | 3 0 0 |
-----------------------
- Create a get_presence(cells) method in the class that takes in any 9 x 9 2D list, representing an unsolved/half-solved puzzle, and returns a sort of indicator regarding whether a given number (between 1 and 9) is present in a given row, column, or quadrant.
For instance, in the preceding example, the returned value of this method should be able to tell you that 2, 3, and 6 are present in the first row, while no number is present in the second column.
- Create a get_possible_values(cells) method in the class that also takes in any 2D list representing an incomplete solution and returns a dictionary, whose keys are the locations of currently empty cells and the corresponding values are the lists/sets of possible values that those cells can take.
These lists of possible values should be generated by taking into account whether a number is present in the same row, column, or quadrant as a given empty cell.
- Create a simple_update(cells) method in the class that takes in any 2D incomplete solution list and calls the get_possible_values() method on that list. From the returned value, if there is an empty cell that holds only one possible solution, update that cell with that value.
If such an update does happen, the method should call itself again to keep updating the cells. This is because after an update, the list of possible values for the remaining empty cells might change. The method should return the updated 2D list in the end.
- Create a recur_solve(cells) method in the class that takes in any 2D incomplete solution list and performs backtracking. First, this method should call simple_update() and return whether or not the puzzle is completely solved (that is, whether or not there are empty cells in the 2D list).
Next, consider the possible values of the remaining empty cells. If there are empty cells remaining and you have no possible values, return a negative result to indicate that we have reached an invalid solution.
On the other hand, if all cells have at least two possible values, find the cell that has the fewest number of possible values. Loop through these possible values, sequentially fill them in the empty cell, and call recur_solve() inside itself with the updated cells to implement the recursive nature of the algorithm. At each iteration, return whether the final solution is valid. If no valid final solution is found via any of the possible values, return a negative result.
- Wrap the preceding methods in a solve() method, which should print out the initial puzzle, pass it to the recur_solve() method, and print out the returned solution from that method.
For example, with the preceding puzzle, a Solver instance, when solve() is called, will print out the following output.
Initial puzzle:
-----------------------
0 0 3 | 0 2 0 | 6 0 0 |
9 0 0 | 3 0 5 | 0 0 1 |
0 0 1 | 8 0 6 | 4 0 0 |
-----------------------
0 0 8 | 1 0 2 | 9 0 0 |
7 0 0 | 0 0 0 | 0 0 8 |
0 0 6 | 7 0 8 | 2 0 0 |
-----------------------
0 0 2 | 6 0 9 | 5 0 0 |
8 0 0 | 2 0 3 | 0 0 9 |
0 0 5 | 0 1 0 | 3 0 0 |
-----------------------
Solved puzzle:
-----------------------
4 8 3 | 9 2 1 | 6 5 7 |
9 6 7 | 3 4 5 | 8 2 1 |
2 5 1 | 8 7 6 | 4 9 3 |
-----------------------
5 4 8 | 1 3 2 | 9 7 6 |
7 2 9 | 5 6 4 | 1 3 8 |
1 3 6 | 7 9 8 | 2 4 5 |
-----------------------
3 7 2 | 6 8 9 | 5 1 4 |
8 1 4 | 2 5 3 | 7 6 9 |
6 9 5 | 4 1 7 | 3 8 2 |
-----------------------
Extensions
1. Go to the Project Euler website, https://projecteuler.net/problem=96, to test out your algorithm against the included puzzles.
2. Write a program that generates Sudoku puzzles and includes unit tests that check whether the solutions generated by our solver are correct.
Note
The solution for this activity can be found on page 648.