Orchestrator

Orchestrator is a Go service running on Imola and Monza boards, that:

• auto-updates itself and other components or data
• manages and runs Arduino Apps on the board (both Linux and microcontroller parts)
• provides multiple APIs to perform actions and fetch data, used by the front-end (Apps Lab)
• handles the communication between the sketch and the Linux part (including Python or containers)
• tunnels TPC sockets to the microcontroller, providing internet connectivity to the sketch

Other useful repositories

• https://github.com/bcmi-labs/app-bricks-py
  Python bricks, containers and examples
• https://github.com/bcmi-labs/app-bricks-example
  Arduino App examples repository
• https://github.com/bcmi-labs/RpcLite
  RPC sketch library (based on MessagePack) to communicate with the Linux part
• https://github.com/bcmi-labs/arduino-deb-images
  Debian image for Imola/Monza boards, ready to be flashed
• https://github.com/bcmi-labs/ArduinoCore-zephyr/tree/imola
  Repository (and branch) containing the arduino:zephyr:unoq platform

How to test locally

Requirements

• Docker, Docker Compose
• adb(eg: sudo apt install adb)

The orchestrator will run on a ARM64 linux system, connected to a microcontroller via SPI.
The service can be developed and tested in two ways:

• on a Linux ARM64 virtual machine, with Debian installed on it (more similar to the real environment)
• locally on your computer, provided that you have a working Docker installed and configured

The communication with the MCU (running the sketch) can be emulated with an Arduino board, connected via serial. The communication with the orchestrator works the same (either with SPI or serial).

How to run the examples

The orchestrator, in order to run apps or examples, needs access to some (currently private) docker images, that are hosted on the GitHub Container Registry. To have access to this registry, you need to authenticate (just once) with the docker login command; here are the instructions on how to run that command.

After being authenticated, you can then setup the local environment and launch Arduino apps:

# Setup the local environment (also copies pre-defined examples).
task arduino-app-cli:build:local
task setup:examples:local

# List available apps or examples, and launch one.
./build/arduino-app-cli app list
./build/arduino-app-cli app start examples/imgdetection-with-ui

Available commands:

• To run (and provision) the app: ./arduino-app-cli app start [app path]
• To retrieve the logs: ./arduino-app-cli app logs [app path]
• To stop the app: ./arduino-app-cli app stop [app path]
• To list the running apps: ./arduino-app-cli app list [app path]
• To run the daemon mode (HTTP api server): ./arduino-app-cli daemon [--port 8080]

To create a new app:

# Create a new app from scratch
./arduino-app-cli app new myapp

# Create an app from an existing one (or example)
./arduino-app-cli app new myapp --from-app=examples/justblink

RPC communication (Linux <--> Sketch)

The communication between the sketch and everything running on Linux uses a RPC procotol based on MessagePack. This is implemented by multiple components:

• a RPCLite library to be used in the sketch
• a RPC router service, that runs on Linux and communicates with the board using the serial port
• MessagePack libraries on the Python part

When running an app that uses the RPC (communication between the sketch and Python code) the RPC router is needed. Currently this component must be started manually with the following command:

go run ./cmd/arduino-router/ -p=port

The port can be taken from the output of the arduino-cli board list command.

ADB file transfer

Files are transferred from and to the board using the adb service.
In order to test this locally, without a board with Linux and adb, you can run a local containerized adbd service with task adbd:start. The service will expose the adbd/testdata path of this repo.

You can use adbfs command cli as follows:

go run ./cmd/arduino-fs-cli ls
go run ./cmd/arduino-fs-cli pull . <some dir>
go run ./cmd/arduino-fs-cli push <some dir> .

There is also a Wails test app, in adbfs-wails folder. You can run it with the command:

task wails:up

Board emulator setup: test locally without having the board

The emulator setup has two docker images:

• a board-emulator container running Debian and systemd with the user arduino.
• a aptrepo-emulator container running nginx that expose the apt packages.

Initial setup of the board:

• task board-emulator:start run the emulators locally

• VERSION=1 ARCH=amd64 task build-deb to build the deb packages

• task board-emulator:debs:install -- arduino-router_1-1_amd64.deb to install the deb packages

• task board-emulator:debs:install -- orchestrator_1-1_amd64.deb to install the deb packages

• task board-emulator:services:start -- arduino-orchestrator to start the orchestrator service.

• task board-emulator:services:start -- arduino-router to start the router (if needed) Test upgrade of deb packages

• task board-emulator:orchestrator:version to check the version of the running arduino-orchestrator. Expected: {"version":"1"}

• VERSION=2 ARCH=amd64 task board-emulator:orchestrator:release create new deb package with a version=2 and publish to the apt repo

• task board-emulator:update:check to check if there are packages to be upgraded. Expected {"packages":[{"name":"arduino-orchestrator","from_version":"1","to_version":"2"}]}

• task board-emulator:update:events to listen for SSE event of the upgrade. Expected (See logs after)

• task board-emulator:update:apply to start the updgrade. Expected Status: 202

• task board-emulator:orchestrator:version. Expected {"version":"2"}

Expected events received in the sse stream after the apply

event: log
data: "Reading package lists..."

event: log
data: "Building dependency tree..."

event: log
data: "Reading state information..."

event: log
data: "Calculating upgrade..."

event: log
data: "The following packages will be upgraded:"

event: log
data: "  arduino-orchestrator"

event: log
data: "1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded."

event: log
data: "Need to get 16.7 MB of archives."

event: log
data: "After this operation, 0 B of additional disk space will be used."

event: log
data: "Get:1 http://aptrepo-emulator:8080 ./ arduino-orchestrator 2 [16.7 MB]"

event: log
data: "Fetched 16.7 MB in 0s (153 MB/s)"

event: log
data: "(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 14515 files and directories currently installed.)\r"

event: log
data: "Preparing to unpack .../arduino-orchestrator_2_amd64.deb ...\r"

event: log
data: "Removed '/etc/systemd/system/multi-user.target.wants/arduino-orchestrator.service'.\r\r"

event: log
data: "fatal: The group `arduino' already exists.\r"

event: log
data: "fatal: The user `arduino' already exists.\r"

event: log
data: "groupadd: group 'sysupgrade' already exists\r"

event: log
data: "Unpacking arduino-orchestrator (2) over (1) ...\r"

event: log
data: "Setting up arduino-orchestrator (2) ...\r"

event: log
data: "Created symlink '/etc/systemd/system/multi-user.target.wants/arduino-orchestrator.service' → '/etc/systemd/system/arduino-orchestrator.service'.\r\r"

event: log
data: ""

event: log
data: "Services to be restarted:"

event: log
data: " systemctl restart arduino-orchestrator.service"

event: log
data: ""

event: log
data: "No containers need to be restarted."

event: log
data: ""

event: log
data: "No user sessions are running outdated binaries."

event: log
data: ""

event: log
data: "No VM guests are running outdated hypervisor (qemu) binaries on this host."

event: restarting
data: "Upgrade completed. Restarting ..."

curl: (18) transfer closed with outstanding read data remaining
task: Failed to run task "board-emulator:update:events": exit status 18

Other low-level commands:

• task board-emulator:debs:list list the available debs packages
• task board-emulator:debs:upgrade -- orchestrator_4-1_amd64.deb to upgrade to version 4 manually
• task board-emulator:needrestart -- -r a run the needrestart command inside the board emulator and restart the needed services
• task board-emulator:services:status -- arduino-orchestrator to check status of a service
• task board-emulator:services:start -- arduino-orchestrator to start a service

Test update of a Wails locally

• task wails:release-server:start to start a local release server listening on port 3001 and serving the public folder with the releases
• task wails:release:local -- 2.0.0 to create a new release 2.0.0
• task wails:up:local to open the wails app click on "Check for Updates" button to start the update from 0.0.0 to 2.0.0.

Orchestrator configurations

The orchestrator will write apps in ~/ArduinoApps folder, and examples in ~/.config/arduino-app-cli/examples. To override the path provide the following flags:

• ARDUINO_APP_CLI__APPS_DIR
• ARDUINO_APP_CLI__CONFIG_DIR

App folder and persistent data

When running an app, persistent files will be saved in the data folder inside the app folder; other supporting files, including the Python venv are saved in the .cache folder inside the app folder.

Docker images registry

Arduino Apps bricks might required a docker image, in that case the orchestrator will pull those from the registry configured with the DOCKER_REGISTRY_BASE environment variable. By default this points to an Arduino AWS Container Registry (public.ecr.aws/arduino).

The only image that needs to be referenced directly is the base Python image (DOCKER_PYTHON_BASE_IMAGE), all other containers can be downloaded automatically by the orchestrator depending on the bricks specified as dependencies in the app.yml file.

Docker configuration on macOS

Docker can be installed with Docker Desktop or colima. In case colima is used, it will not use the default docker context, but a new one called colima. In order for the orchestrator to correctly connect to the docker engine, you need to make sure you have an environment variable pointing to the active docker context socket. For example:

export DOCKER_HOST=unix:///Users/username/.colima/default/docker.sock

If this is not done, you might get errors in the orchestrator, in the form of:

Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?