KCIDB contribuition #KCIDB


Santiago.Esteban via info
 

Good morning KernelCI, Nikolai,

As I mention here before, I work at Microchip, and we are building a
test farm based on Labgrid which currently, allow us to test Linux
kernels in several ARM SoC devices (Cortex-A5, ARM9 variants). We would
like to contribute to KernelCI effort with our builds/tests but support
for non Lava external labs is not available yet.

Therefore, It seems that KCIDB is the place for us. We would like to
start contributing our builds/tests to this project.

As part of our current efforts, we have build a local instance of the
KernelCI infrastructure. I've already asked Nicolai @DevConf, but here,
at a wider audience, Are there any tool that allows to submmit 
"standard" KernelCI formatted data into the KCIDB database?


Best regards,

Santiago Esteban

ps. Nikolai, I liked a lot your presentation @DevConf, it helped me to
understand some aspect of KernelCI  and KCIDB to me.


Nikolai Kondrashov
 

Hello Santiago,

On 2/22/21 10:02 AM, Santiago.Esteban via info via groups.io wrote:
Good morning KernelCI, Nikolai,
As I mention here before, I work at Microchip, and we are building a
test farm based on Labgrid which currently, allow us to test Linux
kernels in several ARM SoC devices (Cortex-A5, ARM9 variants). We would
like to contribute to KernelCI effort with our builds/tests but support
for non Lava external labs is not available yet.
Therefore, It seems that KCIDB is the place for us. We would like to
start contributing our builds/tests to this project.
As part of our current efforts, we have build a local instance of the
KernelCI infrastructure. I've already asked Nicolai @DevConf, but here,
at a wider audience, Are there any tool that allows to submmit
"standard" KernelCI formatted data into the KCIDB database?
I suppose if you already drive everything with an instance of KernelCI
infrastructure, you can just use the KCIDB interface code that was merged
there. However, I think Guillaume would be able to help you better here.

Guillaume, do you think that would work?

Best regards,
Santiago Esteban
ps. Nikolai, I liked a lot your presentation @DevConf, it helped me to
understand some aspect of KernelCI  and KCIDB to me.
Ah, glad to hear that, thanks :)

Nick


Guillaume Tucker
 

On 22/02/2021 09:41, Nikolai Kondrashov wrote:
Hello Santiago,

On 2/22/21 10:02 AM, Santiago.Esteban via info via groups.io wrote:
Good morning KernelCI, Nikolai,

As I mention here before, I work at Microchip, and we are building a
test farm based on Labgrid which currently, allow us to test Linux
kernels in several ARM SoC devices (Cortex-A5, ARM9 variants). We would
like to contribute to KernelCI effort with our builds/tests but support
for non Lava external labs is not available yet.

Therefore, It seems that KCIDB is the place for us. We would like to
start contributing our builds/tests to this project.

As part of our current efforts, we have build a local instance of the
KernelCI infrastructure. I've already asked Nicolai @DevConf, but here,
at a wider audience, Are there any tool that allows to submmit
"standard" KernelCI formatted data into the KCIDB database?
I suppose if you already drive everything with an instance of KernelCI
infrastructure, you can just use the KCIDB interface code that was merged
there. However, I think Guillaume would be able to help you better here.

Guillaume, do you think that would work?
The production instance of kernelci.org is not submitting data to
KCIDB yet because of the overhead related to opening BigQuery
connections. Until the work to use the streaming interface with
a persistent connection is implemented, and potentially a more
longer-term implementation is in place (i.e. not doing it in
kernelci-backend), I would not recommend doing that.

However, if you are available to help it would be great to speed
up the work with redesigning kernelci-backend. It should have
happened about a year ago but there have been many distractions
on the way slowing things down. It's becoming the main sticking
point now, so it will soon be receiving the attention it
deserves.

One thing I don't quite get, is that if you already have an
instance of KernelCI driving your test lab, why not have your
test lab connected to the kernelci.org instance in the same way?
Having Labgrid properly supported by KernelCI is still on the
roadmap, in fact it's related to the redesign of
kernelci-backend.

Having your own CI pipeline is mostly useful if you want to build
your own kernels or run tests in a private environment. All the
current KCIDB contributions are coming from preexisting CI
systems which do just that (distro kernels, private test labs).
But if you want to use the builds from kernelci.org and do what
all the other labs on kernelci.org already do, then using KCIDB
directly doesn't seem like the logical choice to me (and results
going to kernelci.org will also end up in KCIDB).

Best wishes,
Guillaume


Nikolai Kondrashov
 

On 2/22/21 12:25 PM, Guillaume Tucker wrote:
On 22/02/2021 09:41, Nikolai Kondrashov wrote:
I suppose if you already drive everything with an instance of KernelCI
infrastructure, you can just use the KCIDB interface code that was merged
there. However, I think Guillaume would be able to help you better here.

Guillaume, do you think that would work?
The production instance of kernelci.org is not submitting data to
KCIDB yet because of the overhead related to opening BigQuery
connections. Until the work to use the streaming interface with
a persistent connection is implemented, and potentially a more
longer-term implementation is in place (i.e. not doing it in
kernelci-backend), I would not recommend doing that.
Yeah, the current implementation would be bogged down by the amount of
results KernelCI production is producing. However, Santiago, how many
builds/tests do you see produced per day? Perhaps it's worth a try
to see how that's handled, and maybe it's not too bad? Meanwhile
Michal could perhaps finish the rework needed for faster streaming?

Nick


Santiago.Esteban via info
 

On 22/2/21 11:25, Guillaume Tucker wrote:
EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe

On 22/02/2021 09:41, Nikolai Kondrashov wrote:
Hello Santiago,

On 2/22/21 10:02 AM, Santiago.Esteban via info via groups.io wrote:
Good morning KernelCI, Nikolai,

As I mention here before, I work at Microchip, and we are building a
test farm based on Labgrid which currently, allow us to test Linux
kernels in several ARM SoC devices (Cortex-A5, ARM9 variants). We would
like to contribute to KernelCI effort with our builds/tests but support
for non Lava external labs is not available yet.

Therefore, It seems that KCIDB is the place for us. We would like to
start contributing our builds/tests to this project.

As part of our current efforts, we have build a local instance of the
KernelCI infrastructure. I've already asked Nicolai @DevConf, but here,
at a wider audience, Are there any tool that allows to submmit
"standard" KernelCI formatted data into the KCIDB database?
I suppose if you already drive everything with an instance of KernelCI
infrastructure, you can just use the KCIDB interface code that was merged
there. However, I think Guillaume would be able to help you better here.

Guillaume, do you think that would work?
The production instance of kernelci.org is not submitting data to
KCIDB yet because of the overhead related to opening BigQuery
connections. Until the work to use the streaming interface with
a persistent connection is implemented, and potentially a more
longer-term implementation is in place (i.e. not doing it in
kernelci-backend), I would not recommend doing that.

However, if you are available to help it would be great to speed
up the work with redesigning kernelci-backend. It should have
happened about a year ago but there have been many distractions
on the way slowing things down. It's becoming the main sticking
point now, so it will soon be receiving the attention it
deserves.

One thing I don't quite get, is that if you already have an
instance of KernelCI driving your test lab, why not have your
test lab connected to the kernelci.org instance in the same way?
Having Labgrid properly supported by KernelCI is still on the
roadmap, in fact it's related to the redesign of
kernelci-backend.
Having your own CI pipeline is mostly useful if you want to build
your own kernels or run tests in a private environment. All the
current KCIDB contributions are coming from preexisting CI
systems which do just that (distro kernels, private test labs).
But if you want to use the builds from kernelci.org and do what
all the other labs on kernelci.org already do, then using KCIDB
directly doesn't seem like the logical choice to me (and results
going to kernelci.org will also end up in KCIDB).
Currently, we build linux-mainline and linux-next along with our
repositories linux4sam (https://github.com/linux4sam). We also use our
Labgrid farm to tests embedded distributions (Buildroot, Yocto and
Openwrt). Soon (I hope), we will include more test for other parts of
our software stack (u-boot, at91bootstrap) and other hardware tests in
our devices/platforms. Our farm contains 7 different boards today, and
it will keep growing.

If all results from KernelCI will end up also in KCIDB (this was not
clear to me), obviously, it does not make sense push my tests in both
databases.  As you know, we can push test results to staging (I'm not
allowed to push my builds), but we do not have means to get notified
when a new builds are available for testing.  That's what is missing
(for Labgrid or other external lab). While we work in this front, I
understood, from Nikolai's last week presentation, that it would be nice
to push also our result to KCIDB too (and it would be fairly easy).
Anyway, for me it is clear now, that connecting my farm to KernelCI is
key, and as a bonus, we'll get KCIDB :-).


Best wishes,
Guillaume
BR,

Santi


Santiago.Esteban via info
 

On 22/2/21 11:31, Nikolai Kondrashov wrote:
EXTERNAL EMAIL: Do not click links or open attachments unless you know
the content is safe

On 2/22/21 12:25 PM, Guillaume Tucker wrote:
On 22/02/2021 09:41, Nikolai Kondrashov wrote:
I suppose if you already drive everything with an instance of KernelCI
infrastructure, you can just use the KCIDB interface code that was
merged
there. However, I think Guillaume would be able to help you better
here.

Guillaume, do you think that would work?
The production instance of kernelci.org is not submitting data to
KCIDB yet because of the overhead related to opening BigQuery
connections.  Until the work to use the streaming interface with
a persistent connection is implemented, and potentially a more
longer-term implementation is in place (i.e. not doing it in
kernelci-backend), I would not recommend doing that.
Yeah, the current implementation would be bogged down by the amount of
results KernelCI production is producing. However, Santiago, how many
builds/tests do you see produced per day? Perhaps it's worth a try
to see how that's handled, and maybe it's not too bad? Meanwhile
Michal could perhaps finish the rework needed for faster streaming?

Nick
Our CI right now focus on daily builds of  linux-next and linux-mainline
repositories with only two targets (sama5_defconf, at91_dt_defconf): 4
builds tested in 7 boards (and two compilers) = 56 results daily.

I guess, not a lot, considering the amount of data that bigger CI's
farms produce.

BR,

Santi


Nikolai Kondrashov
 

On 2/24/21 11:42 AM, Santiago.Esteban via info via groups.io wrote:
On 22/2/21 11:31, Nikolai Kondrashov wrote:
EXTERNAL EMAIL: Do not click links or open attachments unless you know
the content is safe

On 2/22/21 12:25 PM, Guillaume Tucker wrote:
On 22/02/2021 09:41, Nikolai Kondrashov wrote:
I suppose if you already drive everything with an instance of KernelCI
infrastructure, you can just use the KCIDB interface code that was
merged
there. However, I think Guillaume would be able to help you better
here.

Guillaume, do you think that would work?
The production instance of kernelci.org is not submitting data to
KCIDB yet because of the overhead related to opening BigQuery
connections.  Until the work to use the streaming interface with
a persistent connection is implemented, and potentially a more
longer-term implementation is in place (i.e. not doing it in
kernelci-backend), I would not recommend doing that.
Yeah, the current implementation would be bogged down by the amount of
results KernelCI production is producing. However, Santiago, how many
builds/tests do you see produced per day? Perhaps it's worth a try
to see how that's handled, and maybe it's not too bad? Meanwhile
Michal could perhaps finish the rework needed for faster streaming?

Nick
Our CI right now focus on daily builds of  linux-next and linux-mainline
repositories with only two targets (sama5_defconf, at91_dt_defconf): 4
builds tested in 7 boards (and two compilers) = 56 results daily.
I guess, not a lot, considering the amount of data that bigger CI's
farms produce.
Yeah, that's not much, and you could probably manage with the current
KCIDB-submission code in kernelci. However, if you're going to have your
lab connected anyway soon, the setup effort might not be worth it.

Nick