If you are visiting this page, it is very likely that the software you maintain has been analyzed by my tinderbox system.
What is a tinderbox?
It is a machine that compiles 24/7 that aims to find build failures, test failures, QA issues and so on in the portage tree.
It can be differentiated into:
– tinderbox
– ci
TINDERBOX:
It compiles the entire portage tree against a particular change like:
– a new version of compiler/libc/linker
– a new C/CXX/LD FLAG
– a different toolchain like clang/llvm/lld
– and so on
In short it uses uncommon but supported settings and looks for breakage.
CI:
It is a continuous integration; the CI system compiles the packages after they have been touched in gentoo.git
The CI system uses a standard set of settings, so if you get a bug report from it, it is very likely that the failure is reproducible for users too.
What are the rules that you may know when you see a report from those systems?
1) The reports are filed automatically.
2) Because of the first, it is not possible for me to set an exact error in the bug summary. Instead a general error is used. SEE POINT 18
3) Because of the above, maintainer is encouraged to set an appropriate summary at its convenience
4) Common additional logs (like test-suite.log testlog.txt CMakeOutput.log CMakeError.log LastTest.log config.log testsuite.log autoconf.out) are automatically attached but before of the first if you need something else please ask for them.
5) If you ask for another log, I have to stop the tinderbox service, so there may be a delay between your request and my reaction.
6) There may be an internal reference between round brackets on the “Discovered on” line. This is for me to understand where that failure was reproduced.
7) If you see ‘ci’ as internal reference after you pushed a fix, it is very probably that the bug still exist, or there is another failure in the same ebuild phase. Please inspect deeply the build log. Point 8 may help you about that.
8) At the beginning of the build log a git SHA of the repository at the time of emerging is provided. For convenience there is a link.
9) To avoid making a separate attachment on bugzilla, at the beginning of the build log there is the ’emerge –info’, please check it DEEPLY to understand the system configuration and what differs respect to a more ‘standard’ system (see also point n°25).
10) If you see a compressed build log, is because the plain text version exceeds the limits on our bugzilla (1MB).
11) This system is not perfect. There may be duplicates or invalid bugs.
12) My best suggestion is try to reproduce the issue on empty stage3 (or docker for convenience).
13) When you close the bug with a resolution different from RESOLVED/FIXED please not be cryptic.
14) If new points will be added, there may be a mention like “Valid from YYYY:MM:DD”
15) Valid from 2021-01-10 If in the emerge history there are dev-lang/python-exec, sys-devel/gcc-config and sys-devel/binutils-config means a test with USE=”-native-symlinks” against the package.
16) Valid from 2021-05-04 At the begin of the build log, the commit SHA that causes the build is pointed
out.
17) Valid from 2021-05-04 At the begin of the log there is also the list of installed packages (qlist –
ICvUSS)
18) Valid from 2021-11-11 If the error matches a known pattern, it will be reported in the Summary of the bug, otherwise a general error is used.
19) Valid from 2021-09-28 When there is an open bug about a package, in case the bug is reproduced with a newer version, a comment on the bug is added and the summary is updated with new new version that reproduces the issue.
20) In case of open bugs regarding test failures, CI will not run tests
21) If you are unsure if your commit fixes the reported issue, please always close the bug (as RESOLVED/TEST-REQUEST if you prefer) so the system will open a new bug in case the problem is still reproducible.
22) Given the amount of handled bugs I cannot answer to all generic questions. QA warnings are produced by portage so if there is anything unclear you may want to ask publicly (on irc?) to reduce the response time.
23) Valid from 2021-04-22 Overlays are supported under both CI and TINDERBOX
24) Valid from 2021-11-23 Bugs can be filed on github trackers in case Overlays have their own tracker there.
25) The NOTE at the end of comment 0, may suggest what’s new in the system. Since the new change is configured per-package via package.env, when there will be a build failure related to a DEPEND package, the NOTE may point you in the wrong direction.
26) Valid from 2022-11-03 The Summary contains a sort of tag like ‘FOO-SYSTEM’. This helps make analisys, statistics and so on about a particular configuration.
The list of bugs since when tags were added:
– MUSL-SYSTEM
– LTO-SYSTEM
– CLANG-STRICTER-SYSTEM
– DASH-SYSTEM
– USR-MERGE-SYSTEM
– LIBARCHIVE-SYSTEM
– HARDENED-SYSTEM
– MUSL-CLANG-SYSTEM
– GCC-13-SYSTEM
– CYTHON3-SYSTEM
How much bugs have been filed?
Here are the stats for per year:
– 2020: 5098 bugs
– 2021: 4818 bugs
– 2022: 5817 bugs
– 2023: 3811 bugs
– 2024: bugs