Topics

Mkosi integration with keepalived

Jóhann B. Guðmundsson
 

Hey maybe I get to create the second topic as well, yay talk about breaking the Ice ;)

So first a bit of introduction to what mkosi is.

Mkosi stands for make operating system image and it is a tool for building clean OS images from an upstream distribution in combination with a fresh build of the project in the local working directory and was originally developed and used to simplify testing of systemd/PID1 

The pull request I will be making is targeted at developers/qa since full blown vm's aimed at production by for example hardware vendors or administrators should be kept in a separate repository ++ such images require somewhat further discussion like which OS, how it's setup, which packages etc. 

From documentation stand points the development contribution guidelines would need to be updated to contain the steps required to setup and activate an github account ( not quite necessary but spoon feeding users saves headache down the line ).

Something along the lines of ( note below is aimed at git cli users after account creation )

=== github start ===
Signup to your github account [1]

Once done go to [2]
Put in the name "Command Line" for the Token Description
Enable all the top scopes and click generate token
Copy the token it generates.
Open a terminal window
vim ~/.netrc or use whatever cli editor floats your boat
machine github.com
login [ github login ]
password [ Token from above ]

machine api.github.com
login [github login ]
password [ token from above ]

Save the changes, and quit the editor.
Set correct permission.
chmod 0600 ~/.netrc in

Install git pull requests ( could use hub or something )

Fedora
dnf install git-pull-request

General python
pip3 install git-pull-request

Create a new branch alias
git config --global alias.nb '!git checkout --track $(git config branch.$(git rev-parse --abbrev-ref HEAD).remote)/$(git rev-parse --abbrev-ref HEAD) -b'

=== github end ===

Install mkosi ( requires systemd based distribution with python 3.5+ and systemd 233+ )

Arch
pacman -S mkosi

Debian
apt-get install -y mkosi

Fedora
dnf install mkosi
....

With the above users should ready to use any github repo which has mkosi upstream support.

So now to simple hacking workflow for the intended audience

git clone https://github.com/acassen/keepalived.git
cd keepalived
# the new branch aliases set before
git nb FIX12345
vim keepalived/vrrp/vrrp.c
git commit -a -m 'Fixes #12345'

# generate the test image
mkosi

# run it
systemd-nspawn -bi keepalived.raw

# or
qemu-system-x86_64 -enable-kvm -m 512 -smp 2 -bios /usr/share/edk2/ovmf/OVMF_CODE.fd -hda image.raw
 
# if all local test pass.
git pull-request

So... feedback,constructive comments or flames let me have it ;)

1. https://help.github.com/en/articles/signing-up-for-a-new-github-account
2. https://github.com/settings/tokens/new

Alexandre Cassen
 

Hello,

I need to read it more deeper, current dev circle is based on personal test env (mainly set of vm). While merging commit Quentin and I are making extensive test before merging when proposed fix can be considered as a major patch.

Please give me a little more time to digest and evaluate your proposal.

regs,

Jóhann B. Guðmundsson
 

Hi

Introducing mkosi directly into the ci pipeline is ofcourse a larger scope as you point out but Mkosi is an valuable local development tool ( that is if you run on any of the primary Linux distribution ), and will make "drive-by patching" with testing easier

Just to clarify you don't need to integrate this into your test infrastructure anymore than you like but this is being used in conjunction with travis and bunch of other continues integration tools that integrate with github and elsewhere.

Quentin can also just create a fork repo for the purpose of testing this and I submit it there so it can be tried and worked out to what extent it could be useful to the project.
Quentin ping how does that sound to you?

Quentin Armitage
 

Jóhann

I'm very happy for you to submit a pull request to my fork of keepalived, and I'll have a look at it. At the moment, I have followed the instructions for setting mkosi builds, but I haven't yet understood what are the benefits of using it, either to the developers or users, and it would help if you could explain that.

Many thanks,
Quentin Armitage

On Sat, 2019-05-25 at 03:34 -0700, Jóhann B. Guðmundsson wrote:
Hi

Introducing mkosi directly into the ci pipeline is ofcourse a larger scope as you point out but Mkosi is an valuable local development tool ( that is if you run on any of the primary Linux distribution ), and will make "drive-by patching" with testing easier

Just to clarify you don't need to integrate this into your test infrastructure anymore than you like but this is being used in conjunction with travis and bunch of other continues integration tools that integrate with github and elsewhere.

Quentin can also just create a fork repo for the purpose of testing this and I submit it there so it can be tried and worked out to what extent it could be useful to the project.
Quentin ping how does that sound to you?

Jóhann B. Guðmundsson
 

The mkosi is still missing from the repository so you for example cant create a patch, apply it then run mkosi for testing purposes from the local copy of the git
repository ( or simply generate test images from the upstream repo, like for duplication purposes ).

So you are limited to generate vm's based on downstream package set ( which could be useful to confirm/duplicate issue downstream ) with something like

mkosi -d fedora -t raw_gpt -b --checksum --password password --package keepalived,openssh-clients,vim -o fedora-test.raw

Then run
systemd-nspawn -b -i fedora-test.raw
or
qemu-system-x86_64 -m 512 -smp 2 -bios /usr/share/ovmf/x64/OVMF_CODE.fd -drive format=raw,file=fedora-test.raw

There was an issue there in which you ( Quentin ) seem to be trying to duplicate on Fedora/Centos/Ubuntu in which I pretty sure it would have helped being able to do something like the above and here below

git clone https://github.com/acassen/keepalived.git
cd keepalived
mkosi -d arch or mkosi -d debian or mkosi -d fedora to create vm's with different linux distributions
to duplicate an issue with current upstream against those downstream distribution.

What mkosi adds for upstream is development/testing and continues integration benefits but the general aim of mkosi is to
is to test and debug project,building images for secure devices,building portable service images,building images for secure VMs and containers

My personal use of it is using development/testing and continues integration and with a minor configuration change @dayjob I can replace the entire infrastructure with another linux distribution ( going from fedora, arch or debian etc ) or jump between releases of current one ( like for example from f29 to f30 ) or use upstream sources directly ( as opposed to downstream package set ) for testing and deployment purposes ( if I want to deploy newer stuff than is packaged downstream ) but generating images for production use vs something intended for upstream development and testing are two entirely different things. ( we dont deploy any of the distro with nm or conman, firewalld/iptables and other outdated or unneeded or unwanted components in server infrastructure )

It's probably best that I submit a patch set against your ( Quentin ) keepalived repo on github then you can apply it and see if it does not make your life easier  and could be introduced into the ci pipeline ( travis etc ) then applied to Alexandre repo if it does ( which i would suspect ) ;)

Quentin Armitage
 

On Mon, 2019-05-27 at 07:04 -0700, Jóhann B. Guðmundsson wrote:


It's probably best that I submit a patch set against your ( Quentin ) keepalived repo on github then you can apply it and see if it does not make your life easier  and could be introduced into the ci pipeline ( travis etc ) then applied to Alexandre repo if it does ( which i would suspect ) ;)

Jóhann,

A patch submitted against my github keepalived repo would be good.

Quentin