1kcbenchrate(1)                   User Commands                  kcbenchrate(1)
2
3
4

NAME

6       kcbenchrate  - Linux kernel compile benchmark, rate edition (EXPERIMEN‐
7       TAL)
8

SYNOPSIS

10       kcbenchrate [options]
11

DESCRIPTION

13       Kcbenchrate compiles a Linux kernel on each CPU  core  in  parallel  to
14       test a system's performance or stability.
15
16       Note:  The  optimal number of workers ('-w') that delivers the best re‐
17       sult depends on the machine being benched.  See the section "ON THE DE‐
18       FAULT NUMBER OF WORKERS" below for details.
19
20       To  get  comparable results from different machines you need to use the
21       exact same operating system on all of them.  There are multiple reasons
22       for this recommendation, but one of the main reasons is: the Linux ver‐
23       sion this benchmark downloads and compiles  depends  on  the  operating
24       system's default compiler.
25
26       If  you choose to ignore this recommendation at least make sure to hard
27       code the Linux version to compile ('-s 5.4'), as for example  compiling
28       5.7 will take longer than 5.4 or 4.19 and thus lead to results one can‐
29       not compare.  Also, make sure the compiler used on the systems you want
30       to compare is from similar, as for example gcc10 will try harder to op‐
31       timize the code than gcc8 or gcc9 and thus take more time for its work.
32
33       Kcbench is accompanied by kcbenchrate.  Both  are  quite  similar,  but
34       work slightly different:
35
36       • kcbench tries to build one kernel as fast as possible.  This approach
37         is called 'speed run' and let's make start multiple compilers jobs in
38         parallel  by  using  'make -j #'.  That way kcbench will use a lot of
39         CPU cores most of the time, except during those few phases where  the
40         Linux kernel build process is singled threaded and thus utilizes just
41         one CPU core.  That for example is the case when vmlinux is linked.
42
43       • kcbenchrate tries to keep all CPU cores busy constantly  by  starting
44         workers  on  all  of them, which each builds one kernel with just one
45         job ('make -j 1').  This approach is called 'rate run'.  It  takes  a
46         lot  longer  to  generate  a result than kcbench; it also needs a lot
47         more storage space, but will utilize the machine and  its  processors
48         better.
49
50   Options
51       -b, --bypass
52              After  starting  a  worker wait just a tenths of a second before
53              launching the next to start all the workers a  lot  faster  than
54              usualy.  This can he useful to create a lot of load quickly, but
55              the benchmark result might be slightly inaccurate due to caching
56              effects.
57
58       -h, --help
59              Show usage.
60
61       -i, --iterations int
62              Determines  the  number of kernels that each worker will compile
63              before the end result it printed.  Default: 2
64
65       -j, --jobs int
66              Number of jobs to use when compiling a kernel('make -j #').
67
68              The default is '1'.
69
70       -m, --modconfig
71              Instead of using a config generated  with  'defconfig'  use  one
72              built  by  'allmodconfig'  and compile modules as well.  Takes a
73              lot longer to compile, which is more suitable for machines  with
74              a lot of fast CPU cores.
75
76       -o, --outputdir dir
77              Use  path  to  compile Linux.  Passes 'O=dir/kcbench-worker/' to
78              make when calling it to compile a kernel; use a temporary direc‐
79              tory if not given.
80
81       -s, --src path|version
82              Look  for  sources  in  path,  ~/.cache/kcbench/linux-version or
83              /usr/share/kcbench/linux-version.  If not found try to  download
84              version automatically unless '--no-download' was specified.
85
86       -v, --verbose
87              Increase verboselevel; option can be given multiple times.
88
89       -w, --workers int
90              Number of workers to use.  Default: Number of CPUs.  The optimal
91              setting will depend on the particual machine.  See  ON  THE  DE‐
92              FAULT NUMBER OF WORKERS for details.
93
94       -V, --version
95              Output program version.
96
97       --cc exec
98              Use exec as target compiler.
99
100       --cross-compile arch
101              EXPERIMENTAL:  Cross  compile the Linux kernel.  Cross compilers
102              for this task are packaged in some  Linux  distribution.   There
103              are  also  pre-compiled compilers available on the internet, for
104              example             here:              https://mirrors.edge.ker
105              nel.org/pub/tools/crosstool/
106
107              Values  of  arch  that kcbench/kcbenchrate understand: arm arm64
108              aarch64 riscv riscv64 powerpc powerpc64 x86_64
109
110              Building for archs not directly supported by kcbench/kcbenchrate
111              should work, too: just export ARCH= and CROSS_COMPILE= just like
112              you would when normally cross compiling a Linux kernel.  Do  not
113              use  '--cross-compile'  in  that  case  and  keep  in  mind that
114              kcbench/kcbenchrate configure the compiled Linux kernel with the
115              make  target  'defconfig'  (or  'allmodconfig',  if  you specify
116              '-m'), which might be unusual for  the  arch  in  question,  but
117              might be good enough for benchmarking purposes.
118
119              Be aware there is a bigger risk running into compile errors (see
120              below) when cross compiling.
121
122       --crosscomp-scheme scheme
123              On Linux distributions that are known to  ship  cross  compilers
124              kcbench/  kcbenchrate  will  assume you want to use those.  This
125              parameter allows to specify one of the various different  naming
126              schemes in cases this automatic detection fails or work you want
127              kcbench/kcbenchrate to find them using a 'generic'  scheme  that
128              should  work  with  compilers from various sources, which is the
129              default on unknown distributions.
130
131              Valid values of scheme: debian fedora generic redhat ubuntu
132
133       --hostcc exec
134              Use exec as host compiler.
135
136       --infinite
137              Run endlessly to create system load.
138
139       --llvm Set LLVM=1 to use clang as compiler and LLVM  utilities  as  GNU
140              binutils substitute.
141
142       --add-make-args string
143              Pass  additional flags found in string to make when creating the
144              config or building the kernel.  This option is meant for experts
145              that want to try unusual things, like specifying a special link‐
146              er (--add-make-args 'LD=ld.lld').
147
148              Use with caution!
149
150       --no-download
151              Never download Linux kernel sources from the web automatically.
152
153       --savefailedlogs path
154              Save log of failed compile runs to path.
155

ON THE DEFAULT NUMBER OF WORKERS

157       The optimal number of workers (-w) in most cases will be  identical  to
158       the  number  of CPU cores in the tested machine, that's why this is the
159       default.  But some systems might be a bit faster if they  are  oversub‐
160       scribed a little.  Others might be quicker if you only utilize the real
161       CPU cores and let the cores idle which are only available  due  to  SMT
162       (Simultaneous  Multi-Threading,  also  called Hyper-threading/HT by In‐
163       tel).
164
165       For details and some results  that  show  unexpected  effects  see  the
166       kcbench man page in the section 'ON THE DEFAULT NUMBER OF JOBS'.
167
168       Ideally  kcbenchrate  would do what kcbench does and try a few settings
169       to narrow down the optimal setting.  As this would take quite  a  while
170       this  exercise  is  left  to the user.  Impatient users should consider
171       finding the optimal number of jobs with kcbench and then try  to  start
172       kernbenchrate  with  as many workers, as it might be a good setting for
173       it as well.  You can also try to experiment with  the  number  of  jobs
174       used  per  worker  (-j),  maybe some machines perform best if you start
175       worker on every second core, but use 2 jobs per worker.
176

ON FAILED RUNS DUE TO COMPILATION ERRORS

178       The compilation is unlikely to fail, as long as you are using a settled
179       GCC  version  to  natively compile the source of a current Linux kernel
180       for popular architectures like ARM, ARM64/Aarch64, or x86_64.  For oth‐
181       er  cases there is a bigger risk that compilation will fail due to fac‐
182       tors outside of what kcbench/kcbenchrate  control.   They  nevertheless
183       try  to  catch  a  few common problems and warn, but they can not catch
184       them all, as there are to many factors involved:
185
186       • Brand new compiler generations are sometimes stricter than their pre‐
187         decessors and thus might fail to compile even the latest Linux kernel
188         version.  You might need to use a pre-release  version  of  the  next
189         Linux kernel release to make it work or simply need to wait until the
190         compiler or kernel developers solve the problem.
191
192       • Distributions enable different compiler features that might  have  an
193         impact  on  the  kernel compilation.  For example gcc9 was capable of
194         compiling Linux 4.19 on many distributions, but started  to  fail  on
195         Ubuntu  19.10 due to a feature that got enabled in its GCC.  Try com‐
196         piling a newer Linux kernel version in this case.
197
198       • Cross compilation increases the risk of running into compile problems
199         in  general, as there are many compilers and architectures our there.
200         That for example is why compiling the Linux kernel for  an  unpopular
201         architecture  is  more  likely to fail due to bugs in the compiler or
202         the Linux kernel sources that nobody had noticed before when the com‐
203         piler  or kernel was released.  This is even more likely to happen if
204         you start kcbench/kcbenchrate with  '-m/--allmodconfig'  to  build  a
205         more complex kernel.
206

HINTS

208       Running  benchmarks  is very tricky.  Here are a few of the aspects you
209       should keep mind when doing so:
210
211       • Do not compare results from  two  different  archs  (like  ARM64  and
212         x86_64);  kcbench/kcbenchrate compile different code in that case, as
213         they will compile a native kernel on each of those archs.   This  can
214         be avoided by cross compiling for a third arch that is not related to
215         any of the archs  compared  (say  RISC-V  when  comparing  ARM64  and
216         x86_64).
217
218       • Unless  you  want to bench compilers do not compare results from dif‐
219         ferent compiler generations, as they will apply  different  optimiza‐
220         tions  techniques.   For example to not compare results from GCC7 and
221         GCC9, as the later optimizes harder and thus will take more time gen‐
222         erating  the code.  That's also why the Linux version compiled by de‐
223         fault depends on the machine's compiler: you sometimes can't  compile
224         older  kernels with the latest compilers anyway, as new compiler gen‐
225         erations often uncover bugs in the Linux kernel source that need  get
226         fixed for compiling to succeed.  For example, when GCC10 was close to
227         release it was incapable of compile the then latest Linux version 5.5
228         in  an  allmodconfig  configuration  due to a bug in the Linux kernel
229         sources.
230
231       • Compiling a Linux kernel scales very well and thus can  utilize  pro‐
232         cessors  quite  well.  But be aware that some parts of the Linux com‐
233         pile process will only use one thread (and thus one  CPU  core),  for
234         example  when  linking  vmlinuz; the other cores will idle meanwhile.
235         The effect on the result will grow with the number of CPU cores.
236
237       If you want to work against  that  consider  using  '-m'  to  build  an
238       allmodconfig  configuration with modules; comping a newer, more complex
239       Linux kernel version can also help.  But the best way to avoid this ef‐
240       fect is by running kcbenchrate.
241
242       • kcbench/kcbenchrate  by  default  set  CCACHE_DISABLE=1  when calling
243         'make' to avoid interference from ccache.
244

EXAMPLES

246       To let kcbenchrate decide everything automatically simply run:
247              $ kcbenchrate
248

RESULTS

250       By default the line you are looking for is this:
251
252       4 workers completed 8 kernels so far (avrg: 1100.75 s/run) with a  rate
253       of 13.08 kernels/hour.
254
255       On this quad-core processor four workers each compiled two kernels.  On
256       average, it took each worker 1100.77 seconds to compile one kernel  im‐
257       age.   With a speed like this the machine can compile 13.08 kernels per
258       hour (3600/1100.75*4).
259

MISSING FEATURES

261       • kcbenchrate lacks something similar to 'kcbench --detailedresults'
262
263       • kcbenchrate takes the results verbatim and does not validate them for
264         saneness.   Thus,  if  for example there is some hiccup in the system
265         that heavily slows down one worker temporary kcbenchrate will neither
266         notice nor tell you.
267

SEE ALSO

269       kcbench(1), time(1)
270

AUTHOR

272       Thorsten Leemhuis <linux [AT] leemhuis [DOT] info>
273
274
275
276Version 0.9                                                     kcbenchrate(1)
Impressum