1MK-LOG-PLAYER(1)      User Contributed Perl Documentation     MK-LOG-PLAYER(1)
2
3
4

NAME

6       mk-log-player - Replay MySQL query logs.
7

SYNOPSIS

9       Usage: mk-log-player [OPTION...] [DSN]
10
11       mk-log-player splits and plays slow log files.
12
13       Split slow.log on Thread_id into 16 session files, save in ./sessions:
14
15         mk-log-player --split Thread_id --session-files 16 --base-dir ./sessions slow.log
16
17       Play all those sessions on host1, save results in ./results:
18
19         mk-log-player --play ./sessions --base-dir ./results h=host1
20
21       Use mk-query-digest to summarize the results:
22
23         mk-query-digest ./results/*
24

RISKS

26       The following section is included to inform users about the potential
27       risks, whether known or unknown, of using this tool.  The two main
28       categories of risks are those created by the nature of the tool (e.g.
29       read-only tools vs. read-write tools) and those created by bugs.
30
31       This tool is meant to load a server as much as possible, for stress-
32       testing purposes.  It is not designed to be used on production servers.
33
34       At the time of this release there is a bug which causes mk-log-player
35       to exceed max open files during "--split".
36
37       The authoritative source for updated information is always the online
38       issue tracking system.  Issues that affect this tool will be marked as
39       such.  You can see a list of such issues at the following URL:
40       <http://www.maatkit.org/bugs/mk-log-player>.
41
42       See also "BUGS" for more information on filing bugs and getting help.
43

DESCRIPTION

45       mk-log-player does two things: it splits MySQL query logs into session
46       files and it plays (executes) queries in session files on a MySQL
47       server.  Only session files can be played; slow logs cannot be played
48       directly without being split.
49
50       A session is a group of queries from the slow log that all share a
51       common attribute, usually Thread_id.  The common attribute is specified
52       with "--split".  Multiple sessions are saved into a single session
53       file.  See "--session-files", "--max-sessions", "--base-file-name" and
54       "--base-dir".  These session files are played with "--play".
55
56       mk-log-player will "--play" session files in parallel using N number of
57       "--threads".  (They're not technically threads, but we call them that
58       anyway.)  Each thread will play all the sessions in its given session
59       files.  The sessions are played as fast as possible--there are no
60       delays--because the goal is to stress-test and load-test the server.
61       So be careful using this script on a production server!
62
63       Each "--play" thread writes its results to a separate file.  These
64       result files are in slow log format so they can be aggregated and
65       summarized with mk-query-digest.  See "OUTPUT".
66

OUTPUT

68       Both "--split" and "--play" have two outputs: status messages printed
69       to STDOUT to let you know what the script is doing, and session or
70       result files written to separate files saved in "--base-dir".  You can
71       suppress all output to STDOUT for each with "--quiet", or increase
72       output with "--verbose".
73
74       The session files written by "--split" are simple text files containing
75       queries grouped into sessions.  For example:
76
77         -- START SESSION 10
78
79         use foo
80
81         SELECT col FROM foo_tbl
82
83       The format of these session files is important: each query must be a
84       single line separated by a single blank line.  And the "-- START
85       SESSION" comment tells mk-log-player where individual sessions begin
86       and end so that "--play" can correctly fake Thread_id in its result
87       files.
88
89       The result files written by "--play" are in slow log format with a
90       minimal header: the only attributes printed are Thread_id, Query_time
91       and Schema.
92

OPTIONS

94       Specify at least one of "--play", "--split" or "--split-random".
95
96       "--play" and "--split" are mutually exclusive.
97
98       This tool accepts additional command-line arguments.  Refer to the
99       "SYNOPSIS" and usage information for details.
100
101       --ask-pass
102           group: Play
103
104           Prompt for a password when connecting to MySQL.
105
106       --base-dir
107           type: string; default: ./
108
109           Base directory for "--split" session files and "--play" result
110           file.
111
112       --base-file-name
113           type: string; default: session
114
115           Base file name for "--split" session files and "--play" result
116           file.
117
118           Each "--split" session file will be saved as
119           <base-file-name>-N.txt, where N is a four digit, zero-padded
120           session ID.  For example: session-0003.txt.
121
122           Each "--play" result file will be saved as
123           <base-file-name>-results-PID.txt, where PID is the process ID of
124           the executing thread.
125
126           All files are saved in "--base-dir".
127
128       --charset
129           short form: -A; type: string; group: Play
130
131           Default character set.  If the value is utf8, sets Perl's binmode
132           on STDOUT to utf8, passes the mysql_enable_utf8 option to
133           DBD::mysql, and runs SET NAMES UTF8 after connecting to MySQL.  Any
134           other value sets binmode on STDOUT without the utf8 layer, and runs
135           SET NAMES after connecting to MySQL.
136
137       --config
138           type: Array
139
140           Read this comma-separated list of config files; if specified, this
141           must be the first option on the command line.
142
143       --defaults-file
144           short form: -F; type: string
145
146           Only read mysql options from the given file.
147
148       --dry-run
149           Print which processes play which session files then exit.
150
151       --filter
152           type: string; group: Split
153
154           Discard "--split" events for which this Perl code doesn't return
155           true.
156
157           This option only works with "--split".
158
159           This option allows you to inject Perl code into the tool to affect
160           how the tool runs.  Usually your code should examine $event to
161           decided whether or not to allow the event.  $event is a hashref of
162           attributes and values of the event being filtered.  Or, your code
163           could add new attribute-value pairs to $event for use by other
164           options that accept event attributes as their value.  You can find
165           an explanation of the structure of $event at
166           <http://code.google.com/p/maatkit/wiki/EventAttributes>.
167
168           There are two ways to supply your code: on the command line or in a
169           file.  If you supply your code on the command line, it is injected
170           into the following subroutine where $filter is your code:
171
172              sub {
173                 MKDEBUG && _d('callback: filter');
174                 my( $event ) = shift;
175                 ( $filter ) && return $event;
176              }
177
178           Therefore you must ensure two things: first, that you correctly
179           escape any special characters that need to be escaped on the
180           command line for your shell, and two, that your code is
181           syntactically valid when injected into the subroutine above.
182
183           Here's an example filter supplied on the command line that discards
184           events that are not SELECT statements:
185
186             --filter '$event->{arg} =~ m/^select/i'
187
188           The second way to supply your code is in a file.  If your code is
189           too complex to be expressed on the command line that results in
190           valid syntax in the subroutine above, then you need to put the code
191           in a file and give the file name as the value to "--filter".  The
192           file should not contain a shebang ("#!/usr/bin/perl") line.  The
193           entire contents of the file is injected into the following
194           subroutine:
195
196              sub {
197                 MKDEBUG && _d('callback: filter');
198                 my( $event ) = shift;
199                 $filter && return $event;
200              }
201
202           That subroutine is almost identical to the one above except your
203           code is not wrapped in parentheses.  This allows you to write
204           multi-line code like:
205
206              my $event_ok;
207              if (...) {
208                 $event_ok = 1;
209              }
210              else {
211                 $event_ok = 0;
212              }
213              $event_ok
214
215           Notice that the last line is not syntactically valid by itself, but
216           it becomes syntactically valid when injected into the subroutine
217           because it becomes:
218
219              $event_ok && return $event;
220
221           If your code doesn't compile, the tool will die with an error.
222           Even if your code compiles, it may crash to tool during runtime if,
223           for example, it tries a pattern match an undefined value.  No
224           safeguards of any kind of provided so code carefully!
225
226       --help
227           Show help and exit.
228
229       --host
230           short form: -h; type: string; group: Play
231
232           Connect to host.
233
234       --iterations
235           type: int; default: 1; group: Play
236
237           How many times each thread should play all its session files.
238
239       --max-sessions
240           type: int; default: 5000000; group: Split
241
242           Maximum number of sessions to "--split".
243
244           By default, "mk-log-player" tries to split every session from the
245           log file.  For huge logs, however, this can result in millions of
246           sessions.  This option causes only the first N number of sessions
247           to be saved.  All sessions after this number are ignored, but
248           sessions split before this number will continue to have their
249           queries split even if those queries appear near the end of the log
250           and after this number has been reached.
251
252       --only-select
253           group: Play
254
255           Play only SELECT and USE queries; ignore all others.
256
257       --password
258           short form: -p; type: string; group: Play
259
260           Password to use when connecting.
261
262       --pid
263           type: string
264
265           Create the given PID file.  The file contains the process ID of the
266           script.  The PID file is removed when the script exits.  Before
267           starting, the script checks if the PID file already exists.  If it
268           does not, then the script creates and writes its own PID to it.  If
269           it does, then the script checks the following: if the file contains
270           a PID and a process is running with that PID, then the script dies;
271           or, if there is no process running with that PID, then the script
272           overwrites the file with its own PID and starts; else, if the file
273           contains no PID, then the script dies.
274
275       --play
276           type: string; group: Play
277
278           Play (execute) session files created by "--split".
279
280           The argument to play must be a comma-separated list of session
281           files created by "--split" or a directory.  If the argument is a
282           directory, ALL files in that directory will be played.
283
284       --port
285           short form: -P; type: int; group: Play
286
287           Port number to use for connection.
288
289       --print
290           group: Play
291
292           Print queries instead of playing them; requires "--play".
293
294           You must also specify "--play" with "--print".  Although the
295           queries will not be executed, "--play" is required to specify which
296           session files to read.
297
298       --quiet
299           short form: -q
300
301           Do not print anything; disables "--verbose".
302
303       --[no]results
304           default: yes
305
306           Print "--play" results to files in "--base-dir".
307
308       --session-files
309           type: int; default: 8; group: Split
310
311           Number of session files to create with "--split".
312
313           The number of session files should either be equal to the number of
314           "--threads" you intend to "--play" or be an even multiple of
315           "--threads".  This number is important for maximum performance
316           because it:
317             * allows each thread to have roughly the same amount of sessions
318           to play
319             * avoids having to open/close many session files
320             * avoids disk IO overhead by doing large sequential reads
321
322           You may want to increase this number beyond "--threads" if each
323           session file becomes too large.  For example, splitting a 20G log
324           into 8 sessions files may yield roughly eight 2G session files.
325
326           See also "--max-sessions".
327
328       --set-vars
329           type: string; group: Play; default: wait_timeout=10000
330
331           Set these MySQL variables.  Immediately after connecting to MySQL,
332           this string will be appended to SET and executed.
333
334       --socket
335           short form: -S; type: string; group: Play
336
337           Socket file to use for connection.
338
339       --split
340           type: string; group: Split
341
342           Split log by given attribute to create session files.
343
344           Valid attributes are any which appear in the log: Thread_id,
345           Schema, etc.
346
347       --split-random
348           group: Split
349
350           Split log without an attribute, write queries round-robin to
351           session files.
352
353           This option, if specified, overrides "--split" and causes the log
354           to be split query-by-query, writing each query to the next session
355           file in round-robin style.  If you don't care about "sessions" and
356           just want to split a lot into N many session files and the relation
357           or order of the queries does not matter, then use this option.
358
359       --threads
360           type: int; default: 2; group: Play
361
362           Number of threads used to play sessions concurrently.
363
364           Specifies the number of parallel processes to run.  The default is
365           2.  On GNU/Linux machines, the default is the number of times
366           'processor' appears in /proc/cpuinfo.  On Windows, the default is
367           read from the environment.  In any case, the default is at least 2,
368           even when there's only a single processor.
369
370           See also "--session-files".
371
372       --type
373           type: string; group: Split
374
375           The type of log to "--split" (default slowlog).  The permitted
376           types are
377
378           binlog
379               Split the output of running "mysqlbinlog" against a binary log
380               file.  Currently, splitting binary logs does not always work
381               well depending on what the binary logs contain.  Be sure to
382               check the session files after splitting to ensure proper
383               "OUTPUT".
384
385               If the binary log contains row-based replication data, you need
386               to run "mysqlbinlog" with options "--base64-output=decode-rows
387               --verbose", else invalid statements will be written to the
388               session files.
389
390           genlog
391               Split a general log file.
392
393           slowlog
394               Split a log file in any variation of MySQL slow-log format.
395
396       --user
397           short form: -u; type: string; group: Play
398
399           User for login if not current user.
400
401       --verbose
402           short form: -v; cumulative: yes; default: 0
403
404           Increase verbosity; can be specified multiple times.
405
406           This option is disabled by "--quiet".
407
408       --version
409           Show version and exit.
410
411       --wait-between-sessions
412           type: array; default: 0; group: Play
413
414           Not implemented yet.
415
416           The wait time is given in seconds with microsecond precision and
417           can be either a single value or a range.  A single value causes an
418           exact wait; example: 0.010 = wait 10 milliseconds.  A range causes
419           a random wait between the given value times; example: 0.001,1 =
420           random wait from 1 millisecond to 1 second.
421
422       --[no]warnings
423           default: no; group: Play
424
425           Print warnings about SQL errors such as invalid queries to STDERR.
426

DSN OPTIONS

428       These DSN options are used to create a DSN.  Each option is given like
429       "option=value".  The options are case-sensitive, so P and p are not the
430       same option.  There cannot be whitespace before or after the "=" and if
431       the value contains whitespace it must be quoted.  DSN options are
432       comma-separated.  See the maatkit manpage for full details.
433
434       •   A
435
436           dsn: charset; copy: yes
437
438           Default character set.
439
440       •   D
441
442           dsn: database; copy: yes
443
444           Default database.
445
446       •   F
447
448           dsn: mysql_read_default_file; copy: yes
449
450           Only read default options from the given file
451
452       •   h
453
454           dsn: host; copy: yes
455
456           Connect to host.
457
458       •   p
459
460           dsn: password; copy: yes
461
462           Password to use when connecting.
463
464       •   P
465
466           dsn: port; copy: yes
467
468           Port number to use for connection.
469
470       •   S
471
472           dsn: mysql_socket; copy: yes
473
474           Socket file to use for connection.
475
476       •   u
477
478           dsn: user; copy: yes
479
480           User for login if not current user.
481

DOWNLOADING

483       You can download Maatkit from Google Code at
484       <http://code.google.com/p/maatkit/>, or you can get any of the tools
485       easily with a command like the following:
486
487          wget http://www.maatkit.org/get/toolname
488          or
489          wget http://www.maatkit.org/trunk/toolname
490
491       Where "toolname" can be replaced with the name (or fragment of a name)
492       of any of the Maatkit tools.  Once downloaded, they're ready to run; no
493       installation is needed.  The first URL gets the latest released version
494       of the tool, and the second gets the latest trunk code from Subversion.
495

ENVIRONMENT

497       The environment variable "MKDEBUG" enables verbose debugging output in
498       all of the Maatkit tools:
499
500          MKDEBUG=1 mk-....
501

SYSTEM REQUIREMENTS

503       You need Perl and some core packages that ought to be installed in any
504       reasonably new version of Perl.
505

BUGS

507       For a list of known bugs see
508       <http://www.maatkit.org/bugs/mk-log-player>.
509
510       Please use Google Code Issues and Groups to report bugs or request
511       support: <http://code.google.com/p/maatkit/>.  You can also join
512       #maatkit on Freenode to discuss Maatkit.
513
514       Please include the complete command-line used to reproduce the problem
515       you are seeing, the version of all MySQL servers involved, the complete
516       output of the tool when run with "--version", and if possible,
517       debugging output produced by running with the "MKDEBUG=1" environment
518       variable.
519

COPYRIGHT, LICENSE AND WARRANTY

521       This program is copyright 2008-2011 Percona Inc.  Feedback and
522       improvements are welcome.
523
524       THIS PROGRAM IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR IMPLIED
525       WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF
526       MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.
527
528       This program is free software; you can redistribute it and/or modify it
529       under the terms of the GNU General Public License as published by the
530       Free Software Foundation, version 2; OR the Perl Artistic License.  On
531       UNIX and similar systems, you can issue `man perlgpl' or `man
532       perlartistic' to read these licenses.
533
534       You should have received a copy of the GNU General Public License along
535       with this program; if not, write to the Free Software Foundation, Inc.,
536       59 Temple Place, Suite 330, Boston, MA  02111-1307  USA.
537

AUTHOR

539       Daniel Nichter
540

ABOUT MAATKIT

542       This tool is part of Maatkit, a toolkit for power users of MySQL.
543       Maatkit was created by Baron Schwartz; Baron and Daniel Nichter are the
544       primary code contributors.  Both are employed by Percona.  Financial
545       support for Maatkit development is primarily provided by Percona and
546       its clients.
547

VERSION

549       This manual page documents Ver 1.0.9 Distrib 7540 $Revision: 7531 $.
550
551
552
553perl v5.34.0                      2021-07-22                  MK-LOG-PLAYER(1)
Impressum