Make osbuilder-controller build BYOI

[jimmykarily]

Currently, osbuilder needs the user to provide an image prepared for kairos. e.g. like those we build with Earthly targets:

kairos/Earthfile

Line 539 in dbacc56

iso:

This mean, a user that wants to create a new flavor, needs to take the base OS image (upstream), apply various changes and then feed it to osbuilder.

We would like to have a better (and easier) way to build kairos images out of upstream OS images.

Desired UX

• The user defines either a dockerfile or a base image. This image doesn't need to have the kairos framework baked in or anything kairos specific.
• osbuilder uses some tool (kaniko?) to build:
  • the user's dockerfile
  • a kairos image based on the one above with all our kairos specific modifications
    Ideally, no registry should be required (kaniko can spit out tarballs, check the link above)
• osbuilder (using luet?) will consume the built image, as a tarball and will create isos and everything else.

Limitations

• We will do our best to detect the user defined OS to choose the best framework image. There will be cases which won't work. But out of the box,a the flavors we currently support should still be supported

Why

• Standardize the kairos builds using our own tooling (and not Earthly + docker + whatever)
• Easier "from scratch" builds (less steps for the user)
• Less tools involved (we already need osbuilder, no need for Earthly)
• Quicker experimentation (just try a new base OS image and see if it works)

[jimmykarily]

Tip:

kaniko can produce tarballed docker images:

docker run -v $PWD:/workspace  -it gcr.io/kaniko-project/executor:latest --dockerfile=/workspace/Dockerfile --context=dir://workspace --destination=jimmykarily/test-kaniko --tar-path=/workspace/myimage.tar --no-push

[mudler]

luet code that does unpack: https://github.com/mudler/luet/blob/13dde527b4be2733c34a999d12d3b5676cc583b1/pkg/helpers/docker/docker.go#L161