User Guide

Indended audience: A MicroPython firmware or test developer who wants to use testbed_micropython.

Summary

testbed_micropython is a wrapper around these tests: https://github.com/micropython/micropython/tree/master/tests/xx.py.

The testbed resposibility is:

  • query all connected tentacles

  • build the firmware required by the connected tentacles and board-variants

  • flash the tentacle if required. Some tentacles have to be flashed multiple times to be able to test board-variants.

  • run all tests against these tentacles

  • collect all build results / test results

Installation

Follow these instructions: https://www.octoprobe.org/testbed_showcase/installation/10_ubuntu.html

Please replace every occurence of testenv_showcase with testenv_micropython.

Usage scenarios

Usage principles

A firmware developer typically has the micropython repo checked out and works within that folder.

These 3 companion programs ease the work:

  • mpremote: Communicate with a board.

  • mpbuild: Build a firmware using docker containers.

  • mptest: Test the firmware. mptest makes intense use of mpremote and mpbuild.

Automated Regression Test - full coverage

  • Goal:

    • As much coverage as possible

    • Run fully unattended

mptest test --micropython-tests=<git/branch>  --firmware-build=<git/branch>

  • mptest will:

    • clone/checkout the required branches

    • build the firmware

    • run the tests

    • collect the results.

Automated Regression Test - limited coverage

  • Goal:

    • As above

    • The testrun should be faster (as a developer might be waiting for the results)

    • The testresult should be limited (as a developer just changed code for one board)

mptest test .. --only-board=RPI_PICO2 --only-test=RUN-TESTS_NET_HOSTED

Working on the firmware: Build and test

  • Goal:

    • The developer is working on the C-code and wants to have test feedback fast.

  • Workflow:

    • The developer has the repo checked out and is working on the code.

    • The current directory is always the micropython git repo.

    • Use mptest to compile and test the firmware.

# The current directory is the micropython git repo
mptest test

This now will build/test the firmware against all connected tentacles.

As the git repo is ‘dirty’ the firmware will ALWAYS be flashed.

# As above, but the tests are taken from another branch
mptest test --micropython-tests=<git/branch>

Working on a test: Flash once, test many times

  • Goal:

    • The developer is improving the tests and wants to have test feedback fast.

    • As build/flash is slow, this step should only be done once.

  • Workflow:

    • The developer has the repo checked out and is working on the test code.

    • The current directory is always the micropthon git repo.

    • Preparation: Build the firmware and flash the tentacles - this is slow!

# Preparation: Compile firmware and flash the tentacles
mptest flash --firmware-build=<git/branch>

This will take a few minutes.

# Run the tests
mptest test --flash-skip

This is fast - the tests will be started immediately!