kernelci.org update - 2020-09-04 #minutes


Guillaume Tucker
 

Technical Steering Committee minutes - 2020-09-01
=================================================

* dt-validation
Broonie using KernelCI containers for his local work - thanks Isaiah!

* Profile-driven optimization
Once the AutoFDO stuff lands in the upstream kernel it is likely that there
will be some benefit from sharing the profiling data between users - once
this gets merged we should look into if/how we can share results (especially
from benchmarking when we get to that) - need to understand what it looks
like.

* LLVM/Clang

* Pull request for containers, broonie will send one for build-configs.yaml
once there’s an official release:
https://github.com/kernelci/kernelci-core/pull/469

* Which versions of clang do we want in production? Currently clang-9 and
clang-10, then should we move to clang-10 and clang-11 or clang-11 only?
Worth checking with the Google people what they’re shipping, possibly could
even use their production versions as stable ones to test against. Mark to
start thread, either mail or on github.
-> https://github.com/kernelci/kernelci-core/issues/488

* Docker Hub

Docker Hub limitations getting introduced on 1st November
https://www.docker.com/blog/scaling-docker-to-serve-millions-more-developers-network-egress/
We don’t actually need to keep pulling in every job, images get updated once
a week. So we could use tagged images and pull only if the tag is not
present, rather than blindly pulling the “latest”.

It’s not entirely clear whether this will be impacting us directly or not,
but it’s still a good idea to avoid doing Docker Hub pulls when we don’t need
them.
-> Alternative: deploy our own registry: (less limitations, more maintenance)
https://docs.docker.com/registry/deploying/

* LPC 2020

* KernelCI notes (feel free to add some):
https://docs.google.com/document/d/1sLFx8UDaMIIiU7gPrauNfp3kBVU_v8nXVNPlMUS9kHA/edit#heading=h.toavnesaivaw
-> Preparing KernelCI blog post next week based on that

* KCIDB and kernelci.org: lessons learned from LPC 2020

It’s a great tool for adding data from sources that are entirely decoupled
f rom kernelci.org, such as Fuego and Gentoo’s gKernelCI, 0-Day etc… (seems
like it’s not as trivial as one would have hoped, but it’s still early
days) The need for “native” tests managed directly by kernelci.org is still
very present. So we can deduce some simple rules as to what should get
connected to KCIDB and what should be added to the kernelci.org pipeline:

* KCIDB: independent CI systems, distros and products using downstream
kernel trees, private hardware…

* Native kernelci.org: upstream trees, tests using builds from
kernelci.org, open source test suites and publicly available hardware
only, community CI service for individual test labs…

All the native kernelci.org results are pushed to KCIDB, which becomes a
superset of that data in terms of diversity although not in terms of
schema. Future alternative KernelCI instances on “member”.kernelci.org
should also have their data pushed to KCIDB, where everything can be
compared. Native kernelci.org tests benefit from a fully integrated
pipeline with added functionality, rather than purely a database:
configuration publicly managed on GitHub, regression tracking, automated
bisection…

* KCIDB BoF

* Submission HOWTO published, could use expansion, perhaps.

* Playground system setup. Could be used for playing around with
reporting. Everything is separate, except Grafana, which just needs
another dataset specified, and the result maillist.

* Got Tim Bird from Sony, Alice Ferrazzi from GKernelCI and Tim Orling from
Intel involved, see details in the shared document with LPC notes.

* Real-Time tests
preempt-rt test plan (based on Daniel Wagner’s existing work)
https://github.com/kernelci/kernelci-core/pull/475
See also: https://github.com/bristot/rtsl
https://linuxplumbersconf.org/event/7/contributions/645/

* KCIDB

* Grafana now preserves specified dataset names across dashboard links to
make experimenting easier.

* Experimenting with parsing JSON streams in Python to support pushing more
than one report to KCIDB tools. The stock module is unable to do this. Made
jq.py support parsing streams, but performance is quarter of the stock json
module, even with added optimizations, so far. Perhaps that's good enough
for the start, though.

* kselftest: no test results on staging.kernelci.org?
-> Fixed, there was an arm64 build breakage in linux-next
-> Adding more devices to run kselftest


Technical Steering Committee minutes - 2020-08-18
=================================================

* User settings file
* Settings file support being added in kernelci-core
* Backend API tokens in k8s secrets, that needs to be stored in a settings
file now.
-> Shared volume, or k8s secrets dumped in settings file? TBD

* LLVM/Clang and config fragments
Should we implement our own thing rather than merge_config.sh?
-> Turns out, LLVM=1 solves the problem as long as we're using all the LLVM
toolchain. The issue remains when ussing CC=clang but GNU for everything
else (linker...)

* Bisection
* Fixed, there was a glitch in Jenkins job parameters after k8s roll-out...
* Please check kernelci-results@groups.io for reports until we enable sending
them to the public lists automatically again

* Generic API for test reuslts in kernelci-backend
* Cleaning up backend test API and kci_data
* Fixed some issues with test counts and cleaned up fields
* Next step is to drop requirements for build-related meta-data (for DT
validation and KUnit…)

* KCIDB
* In-object gradual reporting in KCIDB
* WIP: Common Reporting article and LPC talk
* Common Reporting BoF approved. Looks like it will be on Thursday.
* Picking up Submission HOWTO again.
* CKI is stretching KCIDB with their requirements. Reporting core dumps and
coverage.

* KUnit
Initial review/testing with PR from Brendan

* LAVA callback support
* LAVA log filtering enabled on staging
* Changes in login test case, WIP

* Other notes:
https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/


Advisory Board minutes - 2020-08-19
===================================

* Public datawarehouse with dummy data for illustration is live:
https://datawarehouse.cki-project.org

* LPC update: testing/fuzzing MC schedule published:
https://linuxplumbersconf.org/event/7/sessions/80/#20200826


Advisory Board minutes - 2020-09-01
===================================

* Metric: pain-free kernel migration. How does KernelCI facilitate that?
* Kernel release report with summary of results from KernelCI?
* Find out which parts break when migrating to a new kernel
-> Guenter: DRM, sound, wireless
-> Guy: what about kernel configuration?
Kernel version changes, but also need to verify that the config is
compatible See also:
-> Guillaume: see also https://github.com/TuxML (was discussed at LPC)

* Discussing LPC results, KernelCI was mentioned in many places


Summary of changes going into production
========================================

* Add support for user settings files, see documentation:
https://github.com/kernelci/kernelci-core/blob/master/doc/settings.md

* Enable kselftest tarball with kernel builds which include the kselftest
fragment (tests not run yet in production)

* Switch to LLVM=1 for LLVM/Clang builds, which fixes config fragments support
with Clang but now uses all LLVM toolchain rather than mixed LLVM/GNU

* Clean up schema for submitting test results in kernelci-backend generic API

* Several more new NXP i.mx6,7,8 boards

* Enable extra linux-next builds with 64K pages on arm64

* Fix Renesas branch name


Best wishes,
Guillaume