docs.coopcloud.tech/docs/abra/hack.md

---
title: Hack
---

## Contributing

Welcome to Hacking the Planet with `abra`! We're looking forward to see what
you come up. If you have any questions, don't hesitate to ask 💖 If any of your
changes seems a bit controversial, it's probably to come have a chat first to
avoid heartache.

In general, we're into the idea of "Optimistic Merging" (instead of
"Pessimistic Merging" based on our understanding of
[C4](https://hintjens.gitbooks.io/social-architecture/content/chapter4.html)
(described further down under "Development Process" and also [in this blog
post](http://hintjens.com/blog:106)).

## Quick start

Get a fresh copy of the `abra` source code from [here](https://git.coopcloud.tech/coop-cloud/abra).

Install [direnv](https://direnv.net), run `cp .envrc.sample .envrc`, then run `direnv allow` in this directory. This will set coopcloud repos as private due to [this bug.](https://git.coopcloud.tech/coop-cloud/coopcloud.tech/issues/20#issuecomment-8201). Or you can run `go env -w GOPRIVATE=coopcloud.tech` but I'm not sure how persistent this is.

Install [Go >= 1.16](https://golang.org/doc/install) and then:

- `make build` to build. If this fails, run `go mod tidy`.
- `./abra` to run commands
- `make test` will run tests
- `make install-abra` will install abra to `$GOPATH/bin`
- `make install-kadabra` will install kadabra to `$GOPATH/bin`
- `go get <package>` and `go mod tidy` to add a new dependency

Our [Drone CI configuration](https://git.coopcloud.tech/coop-cloud/abra/src/branch/main/.drone.yml) runs a number of checks on each pushed commit. See the [Makefile](https://git.coopcloud.tech/coop-cloud/abra/src/branch/main/Makefile) for more handy targets.

Please use the [conventional commit format](https://www.conventionalcommits.org/en/v1.0.0/) for your commits so we can automate our change log.

## Unit tests

### Run tests

Run the entire suite.

```
make test
```

### Filter tests

Run a specific test.

```
go test ./pkg/recipe -v -run TestGetVersionLabelLocalDoesNotUseTimeoutLabel
```

## Integration tests

### Install dependencies

We use [`bats`](https://bats-core.readthedocs.io/en/stable/), you can install
the required dependencies with the following. You also need a working
installation of Docker and Go (not covered in this section).

```
apt install bats-file bats-assert bats-support jq make git
```

Unfortunately, the latest `bats` version in Debian stable does not have the
"filter tests by tags" feature, which is very handy for running a subset of the
tests. For this, we need to install `bats` from source. It's easy.

```
apt purge -y bats
git clone https://github.com/bats-core/bats-core.git
cd bats-core
sudo ./install.sh /usr/local
```

### Setup Test Server

For many tests an actual server is needed, where apps can be deployed. You can
either use a local one or a remote test server.

#### With remote test server

```
export ABRA_TEST_DOMAIN="test.example.com"
export ABRA_DIR="$HOME/.abra_test"
```

`ABRA_TEST_DOMAIN` should also have a DNS A record for `*.test.example.com`
which points to the same server so that the test suite can deploy apps freely.
It's advised that you re-use the same server and therefore the same Traefik
deployment for running your integration tests. The test suite does not deploy
Traefik for you. Then you'll have more stable results.

You probably don't want to run the entire test suite though, it takes a while.
Try the following for starters.

#### With local swarm

When running the test suite localy you need a running docker swarm setup:

```
docker swarm init
docker network create -d overlay proxy
```

To use the local swarm set the foloowing env var:

```
export TEST_SERVER=default
export ABRA_DIR="$HOME/.abra_test"
```

### Run tests

Now you can run the whole test suite:

```
bats -Tp tests/integration
```

Or you can run a single test file:

```
bats -Tp tests/integration/autocomplete.bats
```

### Tagging tests

When a test actually deploys something to a server, we tag it with the following:

```
# bats test_tags=slow
@test "..." {
  ...
}
```

Then we can use [filters](#filter-tests) (see below) to pick out a subset of
tests which do/do not use a live server. Feel free to come up with your own
tags. See the `bats-core`
[docs](https://bats-core.readthedocs.io/en/stable/writing-tests.html#tagging-tests)
for more.

### Filter tests

You can run a specific file.

```
bats -Tp tests/integration/autocomplete.bats
```

For example, if you want to check that all `abra recipe ...` tests remain working.

```
bats -Tp tests/integration/recipe_*
```

You can filter on test names to run specific kinds of tests.

```
bats -Tp tests/integration --filter "validate app argument"
```

You can filter on tags.

```
bats -Tp tests/integration --filter-tags "\!slow" # only fast tests
bats -Tp tests/integration --filter-tags "slow"   # only slow tests
```

You can also only run the previously failed tests.

```
bats -TP tests/integration --filter-status failed
```

### Debug tests

If you're running into issues and want to debug stuff, you can pass `-x` to
`bats` to trace all commands run in the test. You can add `echo '...' >&3`
debug statements to your test to output stuff also.

## Using the `abra` public API

Warning, there is currently no stability promise for the `abra` public API! Most of the internals are exposed in order to allow a free hand for developers to try build stuff. If people start to build things then we can start the discussion on what is useful to have open/closed and keep stable etc. Please let us know if you depend on the APIs!

The `pkg.go.dev` documentation is [here](https://pkg.go.dev/coopcloud.tech/abra). Here's a brief example to get you going:

```go
package main

import (
	"context"
	"fmt"
	"log"

	abraClient "coopcloud.tech/abra/pkg/client"
	dockerClient "github.com/docker/docker/client"
)

func getClient(serverName string) (*dockerClient.Client, error) {
	cl, err := abraClient.New(serverName)
	if err != nil {
		return nil, fmt.Errorf("getClient: %s", err)
	}
	return cl, nil
}

func main() {
	cl, err := getClient("foo.example.com")
	if err != nil {
		log.Fatal(err)
	}

    // do stuff with the client...
    // https://pkg.go.dev/github.com/docker/docker/client
}
```

Some tools that are making use of the API so far are:

* [`kadabra`](https://git.coopcloud.tech/coop-cloud/abra/src/branch/main/cmd/kadabra/main.go)

## Cross-compiling

If there's no official release for the architecture you use, you can cross-compile `abra` very easily. Clone the source code from [here](https://git.coopcloud.tech/coop-cloud/abra) and then:

- enter the `abra` directory
- run `git tag -l` to see the list of tags, choose the latest one
- run `git checkout <tag>`, where `<tag>` is the latest version
- run `GOOS=<os> GOARCH=<arch> [GOARM=<arm>] make build`. You only have to use `GOARM` if you're building for ARM, this specifies the ARM version (5,6,7 etc). See [this](https://go.dev/doc/install/source#environment) for a list of all supported OS'es and architectures.

## Building in Docker

If you are living under a curse of constant Go environment problems, it might be easier to build `abra` using Docker:

```
sudo setenforce 0  # SELinux probably won't allow Docker to access files
docker run -it -v $PWD:/abra golang:1.19.6 bash
cd /abra
. .envrc
git config --global --add safe.directory /abra  # work around funky file permissions
make build
```

## Release management

We use [goreleaser](https://goreleaser.com) to help us automate releases. We use [semver](https://semver.org) for versioning all releases of the tool. While we are still in the public beta release phase, we will maintain a `0.y.z-beta` format. Change logs are generated from our commit logs. We are still working this out and aim to refine our release praxis as we go.

For developers, while using this `-beta` format, the `y` part is the "major" version part. So, if you make breaking changes, you increment that and _not_ the `x` part. So, if you're on `0.1.0-beta`, then you'd go to `0.1.1-beta` for a backwards compatible change and `0.2.0-beta` for a backwards incompatible change.

### Making a new release

- Run the [integration test suite](#integration-tests) and the unit tests (`make test`) (takes a while!)
- Change `ABRA_VERSION` in [`scripts/installer/installer`](https://git.coopcloud.tech/coop-cloud/abra/src/branch/main/scripts/installer/installer) to match the new tag (use [semver](https://semver.org))
- Commit that change (e.g. `git commit -m 'chore: publish next tag x.y.z-beta'`)
- Make a new tag (e.g. `git tag -a x.y.z-beta`)
- Push the new tag (e.g. `git push && git push --tags`)
- Wait until the build finishes on [build.coopcloud.tech](https://build.coopcloud.tech/coop-cloud/abra)
- Deploy the new installer script (e.g. `cd ./scripts/installer && make`)
- Check the release worked, (e.g. `abra upgrade; abra -v`)

## Fork maintenance

### `godotenv`

We maintain a fork of [godotenv](https://git.coopcloud.tech/coop-cloud/godotenv) because we need inline comment parsing for environment files. You can upgrade the version here by running `go get git.coopcloud.tech/coop-cloud/godotenv@0<COMMID>` where `<commit>` is the latest commit you want to pin to. See [`abra#391`](https://git.coopcloud.tech/coop-cloud/abra/pulls/391) for more.

### `docker/client`

A number of modules in [pkg/upstream](https://git.coopcloud.tech/coop-cloud/abra/src/branch/main/pkg/upstream) are copy/pasta'd from the upstream [docker/docker/client](https://pkg.go.dev/github.com/docker/docker/client). We had to do this because upstream are not exposing their API as public.

### `github.com/schultz-is/passgen`

Due to [`coop-cloud/organising#358`](https://git.coopcloud.tech/coop-cloud/organising/issues/358).