Testing

A lot of times, we will look at tests to better understand the original code written.

This goes alongside the philosophy of Test-Driven Development.

This is really insightful from Dijkstra, that I learned from SE212: “Testing shows the presence, not the absence of bugs”.

Testing can only check the program for some (carefully chosen) input values. This is why Formal Verification exists, to check correctness for all possible input values using symbolic reasoning. However, Formal Verification takes a lot more effort.

Types of Tests

There are four different types of tests, each depending on the granularity of code being tested, as well as the goal of the test.

• Unit Test
• Feature Test
• Integration Test
• Performance Test

Tips for testing

• Just one thing per test. → each test focuses on one tiny bit of functionality
• Independence is imperative. Each test unit must be fully independent: able to run alone, and also within the test suite, regardless of the order they are called. The implication of this rule is that each test must be loaded with a fresh dataset and may have to do some cleanup afterward. This is usually handled by setUp() and tearDown() methods.
• Precision is better than parsimony. Use long and descriptive names for testing functions. You should have names such as test_square_of_number_2() or test_square_negative_number()
• Try hard to make tests that are fast
  • In some cases, tests can’t be fast because they need a complex data structure to work on → Keep these heavier tests in a separate test suite that is run by some scheduled task, and run all other tests as often as needed.
• RTMF (Read the manual, friend!). Learn your tools and learn how to run a single test or a test case. Then, when developing a function inside a module, run this function’s tests often, ideally automatically when you save the code.
• Test everything when you start—and again when you finish. Always run the full test suite before a coding session, and run it again after. This will give you more confidence that you did not break anything in the rest of the code.
• Version control automation hooks are fantastic. It is a good idea to implement a hook that runs all tests before pushing code to a shared repository. You can directly add hooks to your version control system, and some IDEs provide ways to do this more simply in their own environments. Continuous Integration
• Write a breaking test if you want to take a break. If you are in the middle of a development session and have to interrupt your work, it is a good idea to write a broken unit test about what you want to develop next. When coming back to work, you will have a pointer to where you were and get back on track faster.
• In the face of ambiguity, debug using a test. The first step when you are debugging your code is to write a new test pinpointing the bug. While it is not always possible to do, those bug catching tests are among the most valuable pieces of code in your project.
• Make tests easy to understand and explain

Automating the Execution of Your Tests

This is closely related to the topic of Continuous Integration. Actually, you should probably just use GitHub actions as I discovered it’s also pretty efficient.

language: python
python:
  - "3.7"
install:
  - pip install -r requirements.txt
script:
  - python rpi/testing/test.py

• Python Testing

SE465

Found some good advice on tests:

• Tests should have a reason to exist. Either have tests that find bugs, or document your behavior. DO NOT WRITE TEST SOLELY OF THE SAKE OF INCREASING CODE COVERAGE.