1kcbenchrate(1) User Commands kcbenchrate(1)
2
6 kcbenchrate - Linux kernel compile benchmark, rate edition (still ex‐
7 perimental)
8
10 kcbenchrate [options]
11
13 On each CPU in a system compile one Linux kernel to benchmark the
14 throughput of a system or test its stability. It will measure how long
15 it took to compile these kernels and will use this to calculate how
16 many kernels the machine can build this way per hour.
17 Kcbenchrate compliments kcbench. Both are quite similar, but work
19 slightly different:
20 · kcbench tries to build one kernel really quickly. This approach is
22 also called 'speed run' and lets make start multiple compilers jobs
23 in parallel by using 'make -j #'. That way kcbench most of the time
24 will use all the CPU cores, except during those few phases where the
25 Linux kernel build process is singled threaded and thus uses just one
26 CPU core. That for example is the case when vmlinux is linked.
27 · kcbenchrate launches a worker on every CPU that complies one kernel
29 with just one job. This approach will utilize the machine and its
30 processors a little better than the speed approach used by kcbench,
31 as it should keep the CPU cores busy all the time, even during phases
32 where the Linux kernel build process uses just one CPU core. Note
33 that this approach takes a lot longer to generate a results and needs
34 a lot more storage space.
35 NOTE: The optimal number of workers ('-w') that delivers the best re‐
37 sult depends on the machine being benched. See the section "ON THE DE‐
38 FAULT NUMBER OF WORKERS" below for details.
39 IMPORTANT: To get comparable results from different machines you should
41 use the exact same operating system on all of them. There are multiple
42 reasons for this recommendation, but one of the main reasons is: the
43 Linux source code automatically downloaded to compile depends on the
44 compiler being used.
45 If you choose to ignore this recommendation at least make sure to hard
47 code the Linux version to compile ('-s 5.4'), as for example compiling
48 5.7 will take longer than 5.4 or 4.19 and thus lead to results one can‐
49 not compare. Also, make sure the compiler used on the systems you want
50 to compare is from similar, as for example gcc10 will try harder to op‐
51 timize the code than gcc8 or gcc9 and thus take more time.
52 Options
54 -b, --bypass
55 After starting a worker wait just a tenths of a second before
56 launching the next to start the workers a lot faster than usu‐
57 aly. This can he useful to create a lot of load quickly, but
58 the benchmark result might be slightly inaccurate due to caching
59 effects.
60 -h, --help
62 Show usage.
63 -i, --iterations int
65 Determines the number of kernels that each worker will compile
66 before the end result it printed. Default: 2
67 -j, --jobs int(,int, int, ...)
69 Number of jobs to use when compiling a kernel('make -j #'). In
70 kcbenchrate the default is '1' and normally should not be
71 changed.
72 -m, --modconfig
74 Instead of using a config generated with 'defconfig' use one
75 built by 'allmodconfig' and compile modules as well. Takes a
76 lot longer to compile, which is more suitable for machines with
77 a lot of fast CPU cores.
78 -o, --outputdir dir
80 Use path to compile Linux. Passes 'O=dir/kcbench-worker/' to
81 make when calling it to compile a kernel; use a temporary direc‐
82 tory if not given.
83 -s, --src path|version
85 Look for sources in path, ~/.cache/kcbench/linux-version or
86 /usr/share/kcbench/linux-version. If not found try to download
87 version automatically unless '--no-download' was specified.
88 -v, --verbose
90 Increase verboselevel; option can be given multiple times.
91 -w, --workers int
93 Number of workers to use. Default: Number of CPUs. The optimal
94 setting will depend on the particual machine. See ON THE DE‐
95 FAULT NUMBER OF WORKERS for details.
96 -V, --version
98 Output program version.
99 --cc exec
101 Use exec as target compiler.
102 --cross-compile arch
104 EXPERIMENTAL: Cross compile the Linux kernel. Cross compilers
105 for this task are packaged in some Linux distribution. There
106 are also for pre-compiled compilers available on the internet,
107 for example here: https://mirrors.edge.ker‐
108 nel.org/pub/tools/crosstool/
109 Values of arch that kcbench/kcbenchrate understand: arm arm64
111 aarch64 riscv riscv64 powerpc powerpc64 x86_64
112 Building for archs not directly supported by kcbench/kcbenchrate
114 should work, too, if you export ARCH= and CROSS_COMPILE= just
115 like you would when normally cross compiling a Linux kernel. Do
116 not use '--cross-compile' in that case and keep in mind that
117 kcbench/kcbenchrate configure the compiled Linux kernel with the
118 make target 'defconfig' (or 'allmodconfig', if you specify
119 '-m'), which might be unusual for the arch in question, but nor‐
120 mally should be good enough for benchmarking.
121 Be aware there is a bigger risk running into compile errors (see
123 below) when cross compiling.
124 --crosscomp-scheme scheme
126 On Linux distributions that are known to ship cross compilers
127 kcbench/kcbenchrate will assume you want to use those. This pa‐
128 rameter allows to use a one of the various different scheme in
129 cases this automatic detection fails or work you want
130 kcbench/kcbenchrate to find them using a 'generic' scheme that
131 should work with compilers from various sources, which is the
132 default on unknown distributions.
133 Valid values of scheme: debian fedora generic redhat ubuntu
135 --hostcc exec
137 Use exec as host compiler.
138 --infinite
140 Run endlessly to create system load.
141 --llvm Set LLVM=1 to use clang as compiler and LLVM utilities as GNU
143 binutils substitute.
144 --add-make-args string
146 Pass additional flags found in string to make when creating the
147 config or building the kernel. This option is meant for experts
148 that want to try unusual things, like specifing a special linker
149 (--add-make-args 'LD=ld.lld'). Use with caution!
150 --no-download
152 Never download Linux kernel sources from the web automatically.
153 --savefailedlogs path
155 Save log of failed compile runs to path.
156
158 The optimal number of workers (-w) depends on the machine being
159 benched. On most systems you will achieve the best result if the num‐
160 ber of workers matches the number of CPU cores, that's why this is the
161 default. But some systems are a bit faster if they are oversubscribed
162 a little. Others are quicker if only the real cores are utilized and
163 those supported thanks to SMT (Simultaneous Multi-Threading, also
164 called Hyper-threading/HT by Intel) are left idle. For details and
165 some results that show this see the kcbench man page in the section ON
166 THE DEFAULT NUMBER OF JOBS. Ideally kcbenchrate would do what kcbench
167 does and try a few settings to narrow down the optimal setting. As
168 this would take quite a while this exercise is left to the user. Impa‐
169 tient users should consider finding the optimal number of jobs with
170 kcbench and and then start kernbenchrate with as many workers, as that
171 likely will lead to a quite good or even the optimal setting for
172 kcbenchrate.
173
175 As long as you are using a settled GCC version to natively compile the
176 source of a current Linux kernel for popular architectures like ARM,
177 ARM64/Aarch64, or x86_64 the compilation is unlikely to fail. For oth‐
178 er cases there is a bigger risk that compilation will fail due to fac‐
179 tors outside of what kcbench/kcbenchrate control. They nevertheless
180 try to catch a few common problems and warn, but they can not catch
181 them all, as there are to many factors involved:
182 · Brand new compiler generations are sometimes stricter than their pre‐
184 decessors and thus might fail to compile even the latest Linux kernel
185 version. You might need to use a pre-release version of the next
186 Linux kernel release to make it work or simply need to wait until the
187 compiler or kernel developers solve the problem.
188 · Distributions enable different compiler features that might have an
190 impact on the kernel compilation. For example gcc9 was capable of
191 compiling Linux 4.19 on many distributions, but started to fail on
192 Ubuntu 19.10 due to a feature that got enabled in its GCC. Compile a
193 newer Linux kernel version in this case.
194 · Cross compilation increases the risk of running into compile problems
196 in general, as there are many compilers and architectures our there.
197 That for example is why compiling the Linux kernel for an unpopular
198 architecture is more likely to fail due to bugs in the compiler or
199 the Linux kernel sources that nobody had noticed before when the com‐
200 piler or kernel was released. This is even more likely to happen if
201 you start kcbench/kcbenchrate with '-m/--allmodconfig' to build a
202 more complex kernel.
203
205 Running benchmarks is very tricky. Here are a few of the aspects you
206 should keep mind when doing so:
207 · kcbench/kcbenchrate by default set CCACHE_DISABLE=1 when calling
209 'make' to avoid interference from ccache.
210 · Do not compare results from two different archs (like ARM64 and
212 x86_64); kcbench/kcbenchrate compile different code in that case, as
213 they will compile a native kernel on each of those archs. This can
214 be avoided by cross compiling for a third arch that is not related to
215 any of the archs compared (say RISC-V when comparing ARM64 and
216 x86_64).
217 · Unless you want to bench compilers do not compare results from dif‐
219 ferent compiler generations, as they will apply different optimiza‐
220 tions techniques. For example to not compare results from GCC7 and
221 GCC9, as the later optimizes harder and thus will take more time gen‐
222 erating the code. That's also why the Linux version compiled by de‐
223 fault depends on the machines compiler: you sometimes can't compile
224 older kernels with the latest compilers anyway, as new compiler gen‐
225 eration often uncover bugs in the Linux kernel source that need get
226 fixed for to make compiling succeed. For example, when GCC10 was
227 close to release it was incapable of compile the then latest Linux
228 version 5.5 in an allmodconfig configuration due to a bug in the Lin‐
229 ux kernel sources.
230 · Compiling a Linux kernel scales very well and thus can utilize pro‐
232 cessors quite well. But be aware that some parts of the Linux com‐
233 pile process will only use one thread (and thus one CPU core), for
234 example when linking vmlinuz; the other cores will idle meanwhile.
235 The effect on the result will grow with the number of CPU cores.
236 If you want to work against that consider using '-m' to build an
238 allmodconfig configuration with modules; comping a newer, more complex
239 Linux kernel version can also help.
240
242 To let kcbenchrate decide everything automatically simply run:
243 $ kcbenchrate
244
246 By default the line you are looking for is this:
247 4 workers completed 8 kernels so far (avrg: 1100.75 s/run) with a rate
249 of 13.08 kernels/hour.
250 On this quad-core processor four workers each compiled two kernels. On
252 average, it took each worker 1100.77 seconds to compile one kernel im‐
253 age. With a speed like this the machine can compile 13.08 kernels per
254 hour (3600/1100.75*4).
255
257 · kcbenchrate lacks something similar to 'kcbench --detailedresults'
258 · kcbenchrate takes the results verbatim and does not validate them for
260 saneness. Thus, if for example there is some hiccup in the system
261 that heavily slows down just one worker then it will neither notice
262 nor tell you.
263
265 kcbench(1), time(1)
266
268 Thorsten Leemhuis <linux [AT] leemhuis [DOT] info>
269Version 0.9 kcbenchrate(1)