dsync89 Wiki

Appearance

BalenaOS ​

BalenaOS is a self-hosted solution for device onboarding.

1. Install openBalena server ​

sudo apt update
apt-get update && apt-get install -y build-essential git docker.io libssl-dev nodejs npm 
sudo systemctl start docker
curl -L https://github.com/docker/compose/releases/download/1.27.4/docker-compose-Linux-x86_64 -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
docker-compose --version

adduser balena
usermod -aG sudo balena
usermod -aG docker balena

su balena
cd /opt
sudo git clone https://github.com/balena-io/open-balena.git
cd open-balena

./scripts/quickstart \
    -U "[email protected]" \
    -P "balena@test" \
    -d dsync89.com

Example output:

balena@gary-VirtualBox:~/open-balena$ ./scripts/quickstart -U [email protected] -P balena@test -d dsync89.com
==> Creating new configuration at: /home/balena/open-balena/config
==> Bootstrapping easy-rsa...
  - Downloading easy-rsa...
==> Generating root CA cert...
==> Generating root cert chain for haproxy...
==> Generating token auth cert...
==> Generating VPN CA, cert and dhparam (this may take a while)...
==> Setting up environment...
==> Adding default compose file...
==> Success!
  - Start the instance with: ./scripts/compose up -d
  - Stop the instance with: ./scripts/compose stop
  - To create a single, flat, docker-compose.yml file, run:

      ./scripts/compose config > docker-compose.yml

  - Use the following certificate with Balena CLI: /home/balena/open-balena/config/certs/root/ca.crt
    IMPORTANT: You will need to restart your Docker daemon after trusting this certificate to allow your workstation to push images to the registry.

Create docker-compose.yml file.

./scripts/compose config > docker-compose.yml

Restart docker.

systemctl restart docker

Start the openBalena server

./scripts/compose up -d

openBalena will run several API servers on the following domains, so we have to add them to the host route.

Add the following openBalena server domains to your host /etc/hosts file.

bash
echo "" >> /etc/hosts
echo "# OpenBalena Endpoints" >> /etc/hosts
echo "127.0.0.1 api.dsync89.com" >> /etc/hosts
echo "127.0.0.1 registry.dsync89.com" >> /etc/hosts
echo "127.0.0.1 vpn.dsync89.com" >> /etc/hosts
echo "127.0.0.1 s3.dsync89.com" >> /etc/hosts

Test ping curl -k https://api.dsync89.com/ping

2. Install BalenaCLI ​

Two options

Option 1: Install at host (standalone) ​

sudo su
mkdir /opt/balena-cli && cd /opt/balena-cli
wget https://github.com/balena-io/balena-cli/releases/download/v12.38.3/balena-cli-v12.38.3-linux-x64-standalone.zip
unzip balena-cli-v12.38.3-linux-x64-standalone.zip
sudo ln -s /opt/balena-cli/balena /usr/local/bin

echo 'balenaUrl: "dsync89.com"' >> $HOME/.balenarc.yml
export NODE_EXTRA_CA_CERTS="/opt/open-balena/config/certs/root/ca.crt"

Login

balena login --credentials --email "[email protected]" --password "balena@test"

Option 2: Install at host (npm compile) ​

cd ~
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.37.2/install.sh | bash

source ~/.bashrc

# use node 10.20, not the latest v15, otherwise will have `primordial not defined` error.
nvm install 10.20.0
nvm use 10.20.0

wget https://github.com/balena-io/balena-cli/archive/v12.37.0.zip
unzip v12.37.0.zip
cd balena-cli-
npm install balena-cli -g --production # append --unsafe-perm if run as root

Troubleshoot

...
npm ERR! npm WARN deprecated [email protected]: core-js@<3 is no longer maintained and not recommended for usage due to the number of issues. Please, upgrade your dependencies to the actual version of core-js@3.
npm ERR! fs.js:52
npm ERR! } = primordials;
npm ERR!     ^
npm ERR!
npm ERR! ReferenceError: primordials is not defined
npm ERR!     at fs.js:52:5
npm ERR!     at req_ (/home/user/.npm/_cacache/tmp/git-clone-7bc0a58c/node_modules/natives/index.js:143:24)
npm ERR!     at Object.req [as require] (/home/user/.npm/_cacache/tmp/git-clone-7bc0a58c/node_modules/natives/index.js:55:10)
npm ERR!     at Object.<anon
...

Solution: Use Node 10, not Node 13 ++

Issue when installing BalenaCLI

$ npm install balena-cli -g --production --unsafe-perm
...
../src/ffi.h:148:19: warning: mangled name for ‘Napi::TypedArray FFI::WrapPointer(Napi::Env, T*, size_t) [with T = void*(void*, const char*) throw ()]’ will change in C++17 because the exception specification is part of a function type [-Wnoexcept-type]
../src/ffi.h:148:19: warning: mangled name for ‘Napi::TypedArray FFI::WrapPointer(Napi::Env, T*, size_t) [with T = int(void*) throw ()]’ will change in C++17 because the exception specification is part of a function type [-Wnoexcept-type]
../src/ffi.h:148:19: warning: mangled name for ‘Napi::TypedArray FFI::WrapPointer(Napi::Env, T*, size_t) [with T = void*(const char*, int) throw ()]’ will change in C++17 because the exception specification is part of a function type [-Wnoexcept-type]
ffi_bindings.target.mk:112: recipe for target 'Release/obj.target/ffi_bindings/src/ffi.o' failed
make: *** [Release/obj.target/ffi_bindings/src/ffi.o] Error 1
make: Leaving directory '/home/user/.nvm/versions/node/v10.16.0/lib/node_modules/balena-cli/node_modules/ffi-napi/build'
gyp ERR! build error
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/home/user/.nvm/versions/node/v10.16.0/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:262:23)
gyp ERR! stack     at ChildProcess.emit (events.js:198:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)
gyp ERR! System Linux 4.9.140-tegra-virt-20201208
gyp ERR! command "/home/user/.nvm/versions/node/v10.16.0/bin/node" "/home/user/.nvm/versions/node/v10.16.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /home/user/.nvm/versions/node/v10.16.0/lib/node_modules/balena-cli/node_modules/ffi-napi
gyp ERR! node -v v10.16.0
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok

> [email protected] install /home/user/.nvm/versions/node/v10.16.0/lib/node_modules/balena-cli/node_modules/drivelist
> prebuild-install || node-gyp rebuild

prebuild-install WARN install No prebuilt binaries found (target=10.16.0 runtime=node arch=arm64 libc= platform=linux)
make: Entering directory '/home/user/.nvm/versions/node/v10.16.0/lib/node_modules/balena-cli/node_modules/drivelist/build'

Option 3: Build BalenaCLI on Container ​

docker build -t cli-ubuntu .

...

Run the script

./balenaos-in-container.sh --image resin/resinos:2.58.6_rev5-jetson-xavier-nx-devkit-emmc --id test -c "$PWD/config.json" --detach

3. Create Device Configuration File for Xavier-NX ​

# create test app and config file
APP="testapp"
#VER="2.60.1+rev"
VER="2.67.3_rev5"
DEV_TYPE="jetson-xavier-nx-devkit-emmc"
balena app create $APP --type $DEV_TYPE
balena config generate \
    --app $APP --version $VER \
    --network ethernet \
    --appUpdatePollInterval 10 \
    --output testapp-config-jetson-xavier-nx-devkit-emmc.json
balena app create hello-world --type jetson-xavier-nx-devkit-emmc

# verify
balena apps

# create onboarding config. Later put to DUT
balena config generate \
--app hello-world \
--version 2.58.6 \
--network ethernet \
--appUpdatePollInterval 10 \
--output hello-world-openbalena-onboard-jetson-xavier-nx-devkit-emmc.json

4. Install balena-os-in-container on Xavier-NX ​

sudo su
cd /opt
git clone https://github.com/balena-os/balenaos-in-container
cd balenaos-in-container

# start in detach container mode, otherwise it will grab docker shell
./balenaos-in-container.sh --image resin/resinos:2.58.6_rev5.dev-jetson-xavier-nx-devkit-emmc --id test1 -c /opt/hello-world-openbalena-onboard-jetson-xavier-nx-devkit-emmc.json --detach

5. Deploy App via BalenaOS ​

dsync89@VirtualBox:~/balena-idle$ balena deploy helloworld --logs
[Info]    No "docker-compose.yml" file found at "/home/gary/balena-idle"
[Info]    Creating default composition with source: "/home/gary/balena-idle"
[Info]    Building for aarch64/jetson-xavier-nx-devkit-emmc
[Build]   Building services...
[Build]   main Preparing...
[Build]   main Step 1/2 : FROM balenalib/jetson-xavier-nx-devkit-emmc-alpine
[Build]   main  ---> df222be26bbc
[Build]   main Step 2/2 : CMD [ "balena-idle" ]
[Build]   main  ---> Running in 3e62603416a3
[Build]   main Removing intermediate container 3e62603416a3
[Build]   main  ---> 9a5a3cb7898d
[Build]   main Successfully built 9a5a3cb7898d
[Build]   main Successfully tagged balena-idle_main:latest
[Build]   main Image size: 67.43 MB
[Build]   Built 1 service in 0:13
[Info]    Creating release...
[Info]    Pushing images to registry...
[Info]    Saving release...
[Success] Deploy succeeded!
[Success] Release: 5747fbb5d58d90638d2ec7cf1ab22aba

			    \
			     \       .,-.@
			      \\   .^xxx-
			       \\,xxxxx;
			        >  xxx/
			    _.-(6'  xx
			   (=___._/` \
			        )  \ |
			       /   / |
			      /    > /
			     j    < _\
			 _.-' :      ``.
			 \ r=._\        `.
			<`\\_  \         .`-.
			 \ r-7  `-. ._  ' .  `\
			  \`,      `-.`7  7)   )
			   \/         \|  \'  / `-._
			              ||    .'
			               \\  (
			                >\  >
			            ,.-' >.'
			           <.'_.'
			             <'

Check logs from devices

dsync89@VirtualBox:~/balena-idle$ balena devices
ID      UUID    DEVICE NAME DEVICE TYPE                  APPLICATION NAME STATUS IS ONLINE SUPERVISOR VERSION OS VERSION           DASHBOARD URL
4100141 eb8e8c8 icy-dew     jetson-xavier-nx-devkit-emmc helloworld       Idle   true      11.14.0            balenaOS 2.58.6+rev5 https://dashboard.balena-cloud.com/devices/eb8e8c86483b33dec90da8cfdccb3902/summary
dsync89@VirtualBox:~/balena-idle$ balena logs eb8e8c8 --tail
[Logs]    [12/22/2020, 6:34:14 AM] Supervisor starting
[Logs]    [12/22/2020, 6:34:20 AM] Applying configuration change {"SUPERVISOR_POLL_INTERVAL":"900000","SUPERVISOR_DELTA_VERSION":"3"}
[Logs]    [12/22/2020, 6:34:20 AM] Applied configuration change {"SUPERVISOR_POLL_INTERVAL":"900000","SUPERVISOR_DELTA_VERSION":"3"}
[Logs]    [12/22/2020, 6:34:21 AM] Creating network 'default'
[Logs]    [12/22/2020, 6:36:15 AM] Supervisor starting
[Logs]    [12/22/2020, 6:36:17 AM] Applying configuration change {"SUPERVISOR_DELTA":"1"}
[Logs]    [12/22/2020, 6:36:17 AM] Applied configuration change {"SUPERVISOR_DELTA":"1"}
[Logs]    [12/22/2020, 7:29:28 AM] Supervisor starting
[Logs]    [12/23/2020, 1:30:04 AM] Creating volume 'resin-data'
[Logs]    [12/23/2020, 1:30:04 AM] Downloading image 'registry2.balena-cloud.com/v2/cc2a8f7ad0b625f26a3dcd0d5d68e202@sha256:99a4507a19859a84cf11e3cc3616bc6300e12a8c3c8021f14714cff5c0526821'
[Logs]    [12/23/2020, 1:30:34 AM] Downloaded image 'registry2.balena-cloud.com/v2/cc2a8f7ad0b625f26a3dcd0d5d68e202@sha256:99a4507a19859a84cf11e3cc3616bc6300e12a8c3c8021f14714cff5c0526821'
[Logs]    [12/23/2020, 1:30:34 AM] Installing service 'main sha256:9a5a3cb7898de85307e19227159a60b6e59442ddeb4d46c4066a7fd3add87264'
[Logs]    [12/23/2020, 1:30:35 AM] Installed service 'main sha256:9a5a3cb7898de85307e19227159a60b6e59442ddeb4d46c4066a7fd3add87264'
[Logs]    [12/23/2020, 1:30:35 AM] Starting service 'main sha256:9a5a3cb7898de85307e19227159a60b6e59442ddeb4d46c4066a7fd3add87264'
[Logs]    [12/23/2020, 1:30:35 AM] Started service 'main sha256:9a5a3cb7898de85307e19227159a60b6e59442ddeb4d46c4066a7fd3add87264'
[Logs]    [12/23/2020, 1:30:35 AM] [main] Idling...

Troubleshooting ​

Modify balenaos-in-container.sh and add the following host entry to connect to our openBalena server.

...
docker_extra_args="--add-host=api.dsync89.com:192.168.10.112 --add-host=registry.dsync89.com:192.168.10.112 --add-host=vpn.dsync89.com:192.168.10.112 --add-host=s3.dsync89.com:192.168.10.112"

Check the list of supported BalenaOS on DUT https://hub.docker.com/r/resin/resinos/tags?page=1&ordering=last_updated&name=xavier

Appendix ​

Keystore ​

{
  "applicationId": 2,
  "deviceType": "jetson-xavier-nx-devkit-emmc",
  "userId": 2,
  "appUpdatePollInterval": 600000,
  "listenPort": 48484,
  "vpnPort": 443,
  "apiEndpoint": "https://api.dsync89.com",
  "vpnEndpoint": "vpn.dsync89.com",
  "registryEndpoint": "registry.dsync89.com",
  "deltaEndpoint": "https://delta.dsync89.com",
  "mixpanelToken": "__unused__",
  "balenaRootCA": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUZZekNDQTB1Z0F3SUJBZ0lVQWJBVzVURHptMnZhY25uUW5GUGF4YU5xS3JRd0RRWUpLb1pJaHZjTkFRRUwKQlFBd0hqRWNNQm9HQTFVRUF3d1RZMkV1Wm1sa2RXTnBZV1ZrWjJVdWQyOXlhekFlRncweU1UQXhNakF3TlRVegpNRGhhRncwek1UQXhNVGd3TlRVek1EaGFNQjR4SERBYUJnTlZCQU1NRTJOaExtWnBaSFZqYVdGbFpHZGxMbmR2CmNtc3dnZ0lpTUEwR0NTcUdTSWIzRFFFQkFRVUFBNElDRHdBd2dnSUtBb0lDQVFDenNmb05hR1RPcERkOWZMNDkKSVNCdWc2c1VHdUh4TzA5bDBaTXpoRytnenluaXpmeWt6Zm9oZFowQkV6bEMycVFycStmSEFyQ2JYQll6ODFzbgpCSWxpbWZrVlFLOEplUFJnbCthV3ZPSjdCYjVPdHdOTlBmZ0YvZDRwN3k2c1YxeVlWaHEyVlE5TDVjYmZScjFYCjlmTGcxTzFaMlBiOXdFREk0akZaUXBsTmxrNTRMWElNWWhvSzdxK2xyQUZJcHB1Z1VYcVVDdlBUS0ZqNG40bmYKcUQvY3N1SjJUL0puWWJEcStFNmR4K1NtV3NTUUcwM09XVFI0ckFQeVlPaFJTNTJORlVlTXR0eFdvMG5LWDBUZwpkTmN1Y0VmOHZmVXEyZm52TUx6a1dPaDlacnlXVEpodGpSc3dnWXIwaHFzak5kZW83RVllUUI2Wkg3eWFPcmFaCjhoV1M1bVJXbUMvNjhPcVNlM1Y3bGpLMzQ3UzU2Wlp2NXY1cUl6WlhZOUNDOU84cTBNLzRtZjZ1KzMxTzdYQkMKbGl3RXhvUFBPTGZLcUl4RTR1Y2dCTnpsMHJhbFhDSm9Oeml1Qm9vVkpYaGZsUmtVL2lUNEp1bzMxUW1qYVJGRgo4SldxSGFHMVRGZzZGNGIrOEFKU2ZqekkzTVBIQnZpSU5wM1d3L2VXMzJsQ0ZPUXJjNXQydzRqbVR5VzU4ZjJ1CkJULzlOcExUak9IUUx3dEczaEFIVFNyemI2Smh5Zk1ZSU9WNFhDSEZYT1JwK3E4SkdoMjNSZVRLZFh2RHdPU0oKN3JJd1VRYlJOUmhZUGk0TDZ6ZHJxRUp4cTlzWG9EbzZiRVYyZFU5SUtYQ2JhbHBuMjZnNG1SdnBZMXpMZ3B0aQo5VTV1VmFySkc3OVg3OER3Y3l0OUE5UEIxUUlEQVFBQm80R1lNSUdWTUIwR0ExVWREZ1FXQkJRbXc4WWxic1RoCkpnTVRoaEFWU1NvSXlyVG95REJaQmdOVkhTTUVVakJRZ0JRbXc4WWxic1RoSmdNVGhoQVZTU29JeXJUb3lLRWkKcENBd0hqRWNNQm9HQTFVRUF3d1RZMkV1Wm1sa2RXTnBZV1ZrWjJVdWQyOXlhNElVQWJBVzVURHptMnZhY25uUQpuRlBheGFOcUtyUXdEQVlEVlIwVEJBVXdBd0VCL3pBTEJnTlZIUThFQkFNQ0FRWXdEUVlKS29aSWh2Y05BUUVMCkJRQURnZ0lCQUsxSUlFMGdpL1ZRZ0R4N3FkSnZhR0dxRklpV1FMZjBpWXV2dkpIajZlcUtzSkpOTWx1dDRFMWgKVW9BODc1MHE2d2NyOTRBdzFoZ21WY1Foamx5WW05WjVMcUZkenBaV28ybU9Jb0V4WGRTU2w1QzhRVWdOaU1yTgoxSzJuenZmV09rRDRucUdnY3JTM0gzbjFZSXp0UkNLTGtpd2VWb3REeGFmTE9RZm1nV0Y1ZCs0TWJDdzUwN3hiCnQ4ZkNjVVY5L0VYUVVYejh3NDAzOGFwaU5xT1ZGbDE2Ty9NVTA1UjhvNkdRNFVILzdoWG1XSDBvUXlMb3JFODAKTkhCUElySVl1bG0ra3hoQU5BQk4rcjF5RWZOcXcvTTZoWHdONnBPZkZqRlYzK2ljV3hDcFJrUHFnVDJjWWQzWApKQk9BR0RzZnFSb3dJUzludWVlZDBSLzVjY1ZnSGZNUy83VnZIY1NEeXAvVDllaDdyU25KVEYyT0psNUcwU2JGCnh1eHJVM0tRdlk1QTdvSTFUMG5OYmJraHlWcFYwREwyNzBkMDRnNzdkNGw3U054RGZyUHVIMVpoNFJhSE92dzkKMGpQSjNwa3MybDNWblEvb1hPZVhnKzlDS1NhZ3ZnQ0w3WisrM1dKcXJyRTZtODBQTHNwcHdSblpsc1k3YXd2UwpNUEVKcmZzOVJjaTcwMmZBSVlEMUlGUE9Ic0V6R05uanNrZGNqZG1NVDdUWmJjL05BWGNOaW9BbTRkdmZ2WXd4CnZkTlhvSUZiUVBZWVZ0bzNIcUp2MVFXR2h3eEw3bFg0YTJNS3lTTUx5enFaWThjc0pkcnl4RjRVUDFHOEVzV2EKemZGSm1yS2JyUDJHYnBqREFDbzcwWVYzd29SVDBaazYydlZCUTdNMjlkemI1ZUNST3JnagotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==",
  "apiKey": "d8HUTMcXYWwEeteUKtbXOKotnkGf0uEl",
  "os": {
    "sshKeys": [
      "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKqztRSge6AqxJ49eZS4ZmN1li2qryNPD+7vByRfGDaU test@balena"
    ]
  }
}

On DUT

./balenaos-in-container.sh --image resin/resinos:2.67.3_rev5.dev-jetson-xavier-nx-devkit-emmc --id test1 -c /opt/openbalena-testapp-config-jetson-xavier-nx-devkit-emmc.json --detach