1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
|
# Project structure
## test-manager
The client part of the testing environment. This program runs on the host and connects over a
virtual serial port to the `test-runner`.
The tests themselves are defined in this package, using the interface provided by `test-runner`.
## test-runner
The server part of the testing environment. This program runs in guest VMs and provides the
`test-manager` with the building blocks (RPCs) needed to create tests.
## test-rpc
A support library for the other two packages. Defines an RPC interface, transports, shared types,
etc.
# Prerequisities
For macOS, the host machine must be macOS. All other platforms assume that the host is Linux.
## All platforms
* Get the latest stable Rust from https://rustup.rs/.
## macOS
Normally, you would use Tart here. It can be installed with Homebrew. You'll also need
`wireguard-tools`, a protobuf compiler, and OpenSSL:
```bash
brew install cirruslabs/cli/tart wireguard-tools pkg-config openssl protobuf
```
### Wireshark
Wireshark is also required. More specifically, you'll need `wireshark-chmodbpf`, which can be found
in the Wireshark installer here: https://www.wireshark.org/download.html
You also need to add the current user to the `access_bpf` group:
```bash
dseditgroup -o edit -a THISUSER -t user access_bpf
```
This lets us monitor traffic on network interfaces without root access.
## Linux
For running tests on Linux and Windows guests, you will need these tools and libraries:
```bash
dnf install git gcc protobuf-devel libpcap-devel qemu \
podman e2tools mingw64-gcc mingw64-winpthreads-static mtools \
golang-github-rootless-containers-rootlesskit slirp4netns dnsmasq \
dbus-devel pkgconf-pkg-config swtpm edk2-ovmf \
wireguard-tools
rustup target add x86_64-pc-windows-gnu
```
# Building the test runner
Building the `test-runner` binary is done with the `build.sh` script.
Currently, only `x86_64` platforms are supported for Windows/Linux and `ARM64` (Apple Silicon) for macOS.
The `build.sh` requires the `$TARGET` environment variable to be set.
For example, building `test-runner` for Linux would look like this:
``` bash
TARGET=x86_64-unknown-linux-gnu ./build.sh
```
## Linux
For a Linux target `podman` is required to build the `test-runner`. See the [Linux section under Prerequisities](#Prerequisities) for more details.
``` bash
TARGET=x86_64-unknown-linux-gnu ./build.sh
```
## macOS
``` bash
TARGET=aarch64-apple-darwin ./build.sh
```
## Windows
The `test-runner` binary for Windows may be cross-compiled from a Linux host.
``` bash
TARGET=x86_64-pc-windows-gnu ./build.sh
```
# Building base images
See [`BUILD_OS_IMAGE.md`](./BUILD_OS_IMAGE.md) for how to build images for running tests on.
# Running tests
See `cargo run --bin test-manager` for details.
## Linux
Here is an example of how to create a new OS configuration and then run all tests:
```bash
# Create or edit configuration
# The image is assumed to contain a test runner service set up as described in ./BUILD_OS_IMAGE.md
cargo run --bin test-manager set debian11 qemu ./os-images/debian11.qcow2 linux \
--package-type deb --architecture x64 \
--artifacts-dir /opt/testing \
--disks ./testrunner-images/linux-test-runner.img
# Try it out to see if it works
cargo run --bin test-manager run-vm debian11
# Run all tests
cargo run --bin test-manager run-tests debian11 \
--display \
--account 0123456789 \
--current-app <git hash or tag> \
--previous-app 2023.2
```
## macOS
Here is an example of how to create a new OS configuration (on Apple Silicon) and then run all
tests:
```bash
# Download some VM image
tart clone ghcr.io/cirruslabs/macos-ventura-base:latest ventura-base
# Create or edit configuration
# Use SSH to deploy the test runner since the image doesn't contain a runner
cargo run --bin test-manager set macos-ventura tart ventura-base macos \
--architecture aarch64 \
--provisioner ssh --ssh-user admin --ssh-password admin
# Try it out to see if it works
#cargo run -p test-manager run-vm macos-ventura
# Run all tests
cargo run --bin test-manager run-tests macos-ventura \
--display \
--account 0123456789 \
--current-app <git hash or tag> \
--previous-app 2023.2
```
## Note on `ci-runtests.sh`
Account tokens are read (newline-delimited) from the path specified by the environment variable
`ACCOUNT_TOKENS`. Round robin is used to select an account for each VM.
|
