Testing & CI/CD
*

* Continuous Integration/Continuous Deployment
  • What types of test are there (testing tools / frameworks)
  • What should you test (why do we test?)
  • How to write testable code
  • What is CI/CD & how to implement it

Software testing

Error Prevention
Syntax error Compiler/Interpreter
Memory error Don't use C / C++
(or use Valgrind, RAII patterns)
Type error Type checker
Logical error Testing

What are logical errors?
Math error
You forget an edge case
Every programmer will make mistakes in their career!

Types of tests

Unit tests

Types of tests

Integration test
Test how parts of the system work together
Catches more mistakes, but is less fine-grained
Set up (a part of) your system, and use it like an 'outsider' would:

Types of tests

End-to-end test
Test how the entire system behaves
Need to set up the system as close to production as possible
Then, use your application as a real user would
Seeks to prevent this problem:

Types of tests

End-to-end test example
In webdev, end-to-end usually means using an automated web browser. Example with Cypress:
Other e2e testing tools include: Playwright, Selenium, Puppeteer
Can be slow, hard to set up, but testing is very thorough

What to test

Backend
Most common mistake: Trusting data you receive
Your own website is not the only thing that can access your backend server
  • What happens if a parameter is None / null?
  • What happens if someone tries an authorized action
    without necessary authorization? Expired authorization?
  • Is the supplied data valid? (e.g. a valid e-mail address)
  • What happens if the same request is sent twice?

Frontend - How to test

Testing UIs
UIs are hard to test
Common strategies include:
  • Unit testing business logic
  • Snapshot testing
  • Component testing

Frontend - What to test

Frontend
If you perform any business logic in the frontend,
isolate the business logic and unit test it
Bad example:
Business logic (shopping cart price) is tied to the UI!

Frontend - What to test

Frontend
If you perform any business logic in the frontend,
isolate the business logic and unit test it
Better example:
Function now only performs logic, no UI updates
However: Unit testing logic in UIs is limited: Cannot test visual bugs

Frontend - How to test

Snapshot testing
Snapshot testing can catch visual regressions
Snapshot tests work like this:
  • Serialize your UI (e.g., to HTML)
  • Save it. This is your UI snapshot.
  • After a change, make a new snapshot and compare
  • Verify that only the parts you wanted to change, changed
Hard to automate, many false positives

Frontend - How to test

Component testing
Like unit tests, but for UIs
Use libraries to mount individual components with mock-data and test them in isolation
Hinges on you using a component framework

Frontend - How to test

Component testing
Example with Vue

Frontend - How to test

End-to-end testing
End-to-end testing also tests the UI
Same caveats still apply: Hard to set up, can be slow
You're also not testing just the UI
So if there's something wrong, you won't know where the mistake is immediately

Testable code

Why is this code hard to test?
Answers:
  1. Non-determinism
  2. Side effects
Better:

Dependency Injection

Classes/functions receive what they need instead of taking it
When running your code while testing,
you would always use the production database...

Dependency Injection

Instead, pass it as a parameter:
When testing, you can now pass a fake / different DbConnection

How to run tests

You could organize all the tests yourself...
Or use a test framework
Python JavaScript JavaScript

Pytest Example

Add a folder called tests
conftest.py contains test configuration and fixtures
Fixtures are reusable pieces of test setup/teardown code
(general term, not Python specific)

Pytest Example

Remember our earlier unit test example?
client here is a fixture

Testing

DRY or not?
Fixtures are nice to avoid needless repetition of setup code
But don't go overboard - Easily readable/searchable test cases are the most important
Repetition in test cases is only natural.
Tolerate it more than you would in normal code

Result

pytest -v --cov-report=html --cov-report=term-missing --cov=src tests/

Continuous Integration

Instead of big releases every now and then, integrate small changes into the codebase continuously
A good set of tests gives you the confidence that your code does not break anything
Workflow:
  1. Write code
  2. Push code
  3. Pull request
  4. Continuous Integration service automatically runs your tests
  5. Merge into master if all the tests succeed

How to integrate tests in your workflow

GitHub Actions is GitHub's built-in Continuous Integration service.
You can use it like this:
  1. Add a folder called .github at the root of your repository
  2. In .github create a folder workflows
  3. In workflows add a file run_tests.yml (or any name you want)
  4. run_tests.yml will contain configuration for the GitHub actions, in YAML format:

Result

Deployment

At some point, you will want to put your website/app/program online
For websites, this means hosting them
Most often, in the Cloud (AWS, GCP, Azure)
Common hosting methods include:
  • Virtual Private Server (VPS)
    > You get a VM and install/run everything
  • Platform-as-a-Service (PaaS)
    > You give your app's repository. It will be placed into a VM with a prebuilt image.
  • Functions-as-a-Service (FaaS)
    > You give your app's functions. Cloud provider invokes them on request and manages scaling automatically.

Deployment

Quick note on hosting
We as LUDev have hosting options available
If you are at the point where you want to have your website online, come talk to us
We have education license with Netlify, but there are other options

Continuous Deployment

Not always possible
Important to have a reproducible build process and very good tests
For now, try to get at least one useful unit test working, and setting up GitHub Actions

The example repositories also have GitHub Actions set up already:

Summary:

  • Decide on a testing strategy
  • Implement it using some tool or framework
  • Automate it with GitHub Actions (CI)