1kcbenchrate(1) User Commands kcbenchrate(1)
2
6 kcbenchrate - Linux kernel compile benchmark, rate edition (EXPERIMEN‐
7 TAL)
8
10 kcbenchrate [options]
11
13 Kcbenchrate compiles a Linux kernel on each CPU core in parallel to
14 test a system's performance or stability.
15 Note: The optimal number of workers ('-w') that delivers the best re‐
17 sult depends on the machine being benched. See the section "ON THE DE‐
18 FAULT NUMBER OF WORKERS" below for details.
19 To get comparable results from different machines you need to use the
21 exact same operating system on all of them. There are multiple reasons
22 for this recommendation, but one of the main reasons is: the Linux ver‐
23 sion this benchmark downloads and compiles depends on the operating
24 system's default compiler.
25 If you choose to ignore this recommendation at least make sure to hard
27 code the Linux version to compile ('-s 5.4'), as for example compiling
28 5.7 will take longer than 5.4 or 4.19 and thus lead to results one can‐
29 not compare. Also, make sure the compiler used on the systems you want
30 to compare is from similar, as for example gcc10 will try harder to op‐
31 timize the code than gcc8 or gcc9 and thus take more time for its work.
32 Kcbench is accompanied by kcbenchrate. Both are quite similar, but
34 work slightly different:
35 • kcbench tries to build one kernel as fast as possible. This approach
37 is called 'speed run' and let's make start multiple compilers jobs in
38 parallel by using 'make -j #'. That way kcbench will use a lot of
39 CPU cores most of the time, except during those few phases where the
40 Linux kernel build process is singled threaded and thus utilizes just
41 one CPU core. That for example is the case when vmlinux is linked.
42 • kcbenchrate tries to keep all CPU cores busy constantly by starting
44 workers on all of them, which each builds one kernel with just one
45 job ('make -j 1'). This approach is called 'rate run'. It takes a
46 lot longer to generate a result than kcbench; it also needs a lot
47 more storage space, but will utilize the machine and its processors
48 better.
49 Options
51 -b, --bypass
52 After starting a worker wait just a tenths of a second before
53 launching the next to start all the workers a lot faster than
54 usualy. This can he useful to create a lot of load quickly, but
55 the benchmark result might be slightly inaccurate due to caching
56 effects.
57 -h, --help
59 Show usage.
60 -i, --iterations int
62 Determines the number of kernels that each worker will compile
63 before the end result it printed. Default: 2
64 -j, --jobs int
66 Number of jobs to use when compiling a kernel('make -j #').
67 The default is '1'.
69 -m, --modconfig
71 Instead of using a config generated with 'defconfig' use one
72 built by 'allmodconfig' and compile modules as well. Takes a
73 lot longer to compile, which is more suitable for machines with
74 a lot of fast CPU cores.
75 -o, --outputdir dir
77 Use path to compile Linux. Passes 'O=dir/kcbench-worker/' to
78 make when calling it to compile a kernel; use a temporary direc‐
79 tory if not given.
80 -s, --src path|version
82 Look for sources in path, ~/.cache/kcbench/linux-version or
83 /usr/share/kcbench/linux-version. If not found try to download
84 version automatically unless '--no-download' was specified.
85 -v, --verbose
87 Increase verboselevel; option can be given multiple times.
88 -w, --workers int
90 Number of workers to use. Default: Number of CPUs. The optimal
91 setting will depend on the particual machine. See ON THE DE‐
92 FAULT NUMBER OF WORKERS for details.
93 -V, --version
95 Output program version.
96 --cc exec
98 Use exec as target compiler.
99 --cross-compile arch
101 EXPERIMENTAL: Cross compile the Linux kernel. Cross compilers
102 for this task are packaged in some Linux distribution. There
103 are also pre-compiled compilers available on the internet, for
104 example here: https://mirrors.edge.ker‐
105 nel.org/pub/tools/crosstool/
106 Values of arch that kcbench/kcbenchrate understand: arm arm64
108 aarch64 riscv riscv64 powerpc powerpc64 x86_64
109 Building for archs not directly supported by kcbench/kcbenchrate
111 should work, too: just export ARCH= and CROSS_COMPILE= just like
112 you would when normally cross compiling a Linux kernel. Do not
113 use '--cross-compile' in that case and keep in mind that
114 kcbench/kcbenchrate configure the compiled Linux kernel with the
115 make target 'defconfig' (or 'allmodconfig', if you specify
116 '-m'), which might be unusual for the arch in question, but
117 might be good enough for benchmarking purposes.
118 Be aware there is a bigger risk running into compile errors (see
120 below) when cross compiling.
121 --crosscomp-scheme scheme
123 On Linux distributions that are known to ship cross compilers
124 kcbench/ kcbenchrate will assume you want to use those. This
125 parameter allows to specify one of the various different naming
126 schemes in cases this automatic detection fails or work you want
127 kcbench/kcbenchrate to find them using a 'generic' scheme that
128 should work with compilers from various sources, which is the
129 default on unknown distributions.
130 Valid values of scheme: debian fedora generic redhat ubuntu
132 --hostcc exec
134 Use exec as host compiler.
135 --infinite
137 Run endlessly to create system load.
138 --llvm Set LLVM=1 to use clang as compiler and LLVM utilities as GNU
140 binutils substitute.
141 --add-make-args string
143 Pass additional flags found in string to make when creating the
144 config or building the kernel. This option is meant for experts
145 that want to try unusual things, like specifying a special link‐
146 er (--add-make-args 'LD=ld.lld').
147 Use with caution!
149 --no-download
151 Never download Linux kernel sources from the web automatically.
152 --savefailedlogs path
154 Save log of failed compile runs to path.
155
157 The optimal number of workers (-w) in most cases will be identical to
158 the number of CPU cores in the tested machine, that's why this is the
159 default. But some systems might be a bit faster if they are oversub‐
160 scribed a little. Others might be quicker if you only utilize the real
161 CPU cores and let the cores idle which are only available due to SMT
162 (Simultaneous Multi-Threading, also called Hyper-threading/HT by In‐
163 tel).
164 For details and some results that show unexpected effects see the
166 kcbench man page in the section 'ON THE DEFAULT NUMBER OF JOBS'.
167 Ideally kcbenchrate would do what kcbench does and try a few settings
169 to narrow down the optimal setting. As this would take quite a while
170 this exercise is left to the user. Impatient users should consider
171 finding the optimal number of jobs with kcbench and then try to start
172 kernbenchrate with as many workers, as it might be a good setting for
173 it as well. You can also try to experiment with the number of jobs
174 used per worker (-j), maybe some machines perform best if you start
175 worker on every second core, but use 2 jobs per worker.
176
178 The compilation is unlikely to fail, as long as you are using a settled
179 GCC version to natively compile the source of a current Linux kernel
180 for popular architectures like ARM, ARM64/Aarch64, or x86_64. For oth‐
181 er cases there is a bigger risk that compilation will fail due to fac‐
182 tors outside of what kcbench/kcbenchrate control. They nevertheless
183 try to catch a few common problems and warn, but they can not catch
184 them all, as there are to many factors involved:
185 • Brand new compiler generations are sometimes stricter than their pre‐
187 decessors and thus might fail to compile even the latest Linux kernel
188 version. You might need to use a pre-release version of the next
189 Linux kernel release to make it work or simply need to wait until the
190 compiler or kernel developers solve the problem.
191 • Distributions enable different compiler features that might have an
193 impact on the kernel compilation. For example gcc9 was capable of
194 compiling Linux 4.19 on many distributions, but started to fail on
195 Ubuntu 19.10 due to a feature that got enabled in its GCC. Try com‐
196 piling a newer Linux kernel version in this case.
197 • Cross compilation increases the risk of running into compile problems
199 in general, as there are many compilers and architectures our there.
200 That for example is why compiling the Linux kernel for an unpopular
201 architecture is more likely to fail due to bugs in the compiler or
202 the Linux kernel sources that nobody had noticed before when the com‐
203 piler or kernel was released. This is even more likely to happen if
204 you start kcbench/kcbenchrate with '-m/--allmodconfig' to build a
205 more complex kernel.
206
208 Running benchmarks is very tricky. Here are a few of the aspects you
209 should keep mind when doing so:
210 • Do not compare results from two different archs (like ARM64 and
212 x86_64); kcbench/kcbenchrate compile different code in that case, as
213 they will compile a native kernel on each of those archs. This can
214 be avoided by cross compiling for a third arch that is not related to
215 any of the archs compared (say RISC-V when comparing ARM64 and
216 x86_64).
217 • Unless you want to bench compilers do not compare results from dif‐
219 ferent compiler generations, as they will apply different optimiza‐
220 tions techniques. For example to not compare results from GCC7 and
221 GCC9, as the later optimizes harder and thus will take more time gen‐
222 erating the code. That's also why the Linux version compiled by de‐
223 fault depends on the machine's compiler: you sometimes can't compile
224 older kernels with the latest compilers anyway, as new compiler gen‐
225 erations often uncover bugs in the Linux kernel source that need get
226 fixed for compiling to succeed. For example, when GCC10 was close to
227 release it was incapable of compile the then latest Linux version 5.5
228 in an allmodconfig configuration due to a bug in the Linux kernel
229 sources.
230 • Compiling a Linux kernel scales very well and thus can utilize pro‐
232 cessors quite well. But be aware that some parts of the Linux com‐
233 pile process will only use one thread (and thus one CPU core), for
234 example when linking vmlinuz; the other cores will idle meanwhile.
235 The effect on the result will grow with the number of CPU cores.
236 If you want to work against that consider using '-m' to build an
238 allmodconfig configuration with modules; comping a newer, more complex
239 Linux kernel version can also help. But the best way to avoid this ef‐
240 fect is by running kcbenchrate.
241 • kcbench/kcbenchrate by default set CCACHE_DISABLE=1 when calling
243 'make' to avoid interference from ccache.
244
246 To let kcbenchrate decide everything automatically simply run:
247 $ kcbenchrate
248
250 By default the line you are looking for is this:
251 4 workers completed 8 kernels so far (avrg: 1100.75 s/run) with a rate
253 of 13.08 kernels/hour.
254 On this quad-core processor four workers each compiled two kernels. On
256 average, it took each worker 1100.77 seconds to compile one kernel im‐
257 age. With a speed like this the machine can compile 13.08 kernels per
258 hour (3600/1100.75*4).
259
261 • kcbenchrate lacks something similar to 'kcbench --detailedresults'
262 • kcbenchrate takes the results verbatim and does not validate them for
264 saneness. Thus, if for example there is some hiccup in the system
265 that heavily slows down one worker temporary kcbenchrate will neither
266 notice nor tell you.
267
269 kcbench(1), time(1)
270
272 Thorsten Leemhuis <linux [AT] leemhuis [DOT] info>
273Version 0.9 kcbenchrate(1)