1PMREORDER(1) PMDK Programmer's Manual PMREORDER(1)
2
6 pmreorder - performs a persistent consistency check using a store re‐
7 ordering mechanism
8
10 $ python pmreorder <options>
11
13 The pmreorder tool is a collection of python scripts designed to parse
14 and replay operations logged by pmemcheck - a persistent memory check‐
15 ing tool.
16 Pmreorder performs the store reordering between persistent memory bar‐
18 riers - a sequence of flush-fence operations. It uses a consistency
19 checking routine provided in the command line options to check whether
20 files are in a consistent state.
21 Considering that logging, replaying and reordering of operations are
23 very time consuming, it is recommended to use as few stores as possible
24 in test workloads.
25
27 -h, --help
28 Prints synopsis and list of options.
30 -l <store_log>, --logfile <store_log>
32 The pmemcheck log file to process.
34 -c <prog|lib>, --checker <prog|lib>
36 Consistency checker type.
38 -p <path>, --path <path>
40 Path to the consistency checker. Checker function has to return 0 for
42 consistent cases and 1 otherwise.
43 -n <name>, --name <name>
45 The symbol name of the consistency checking function in the library.
47 Valid only if the checker type is lib.
48 -o <pmreorder_output>, --output <pmreorder_output>
50 Set the logger output file.
52 -e <debug|info|warning|error|critical>, --output-level <debug|in‐
54 fo|warning|error|critical>
55 Set the output log level.
57 -r <NoReorderNoCheck| NoReorderDoCheck| ReorderFull| ReorderPartial|
59 ReorderAccumulative| ReorderReverseAccumulative>, --default-en‐
60 gine <NoReorderNoCheck| NoReorderDoCheck| ReorderFull| ReorderPartial|
61 ReorderAccumulative| ReorderReverseAccumulative>
62 Set the initial reorder engine. Default value is NoReorderNoCheck.
64 -x <cli_macros|config_file>, --extended-macros <cli_macros|config_file>
66 Assign an engine types to the defined marker.
68
70 By default, the NoReorderNoCheck engine is used, which means that for
71 each set of stores, the tool will pass-through all sequences of stores
72 not reordered and will not run consistency checker on them.
73 To enable different types of the reorder engine and begin proper re‐
75 ordering tests, a number of other engines exist:
76 · NoReorderDoCheck - pass-through of unchanged operations. Checks cor‐
78 rectness of the stores as they were logged. Useful for operations
79 that do not require fail safety.
80 Example:
82 input: (a, b, c)
83 output: (a, b, c)
84 · ReorderAccumulative - checks correctness on a growing subset of the
86 original sequence.
87 Example:
89 input: (a, b, c)
90 output:
91 ()
92 (a)
93 (a, b)
94 (a, b, c)
95 · ReorderReverseAccumulative - checks correctness on a reverted growing
97 subset of the original sequence.
98 Example:
100 input: (a, b, c)
101 output:
102 ()
103 (c)
104 (c, b)
105 (c, b, a)
106 · ReorderPartial - checks consistency on 3 randomly selected sequences
108 from a set of 1000 combinations of the original log, without repeti‐
109 tions.
110 Example:
112 input: (a, b, c)
113 output:
114 (b, c)
115 (b)
116 (a, b, c)
117 · ReorderFull - for each set of stores generates and checks consistency
119 of all possible store permutations. This might prove to be very com‐
120 putationally expensive for most workloads. It can be useful for
121 critical sections of code with limited number of stores.
122 Example:
124 input: (a, b, c)
125 output:
126 ()
127 (a)
128 (b)
129 (c)
130 (a, b)
131 (a, c)
132 (b, a)
133 (b, c)
134 (c, a)
135 (c, b)
136 (a, b, c)
137 (a, c, b)
138 (b, a, c)
139 (b, c, a)
140 (c, a, b)
141 (c, b, a)
142 When the engine is passed with an -r option, it will be used for each
144 logged set of stores. Additionally, the -x parameter can be used to
145 switch engines separately for any marked code sections. For more de‐
146 tails about -x extended macros functionality see section INSTRUMENTA‐
147 TION below.
148
150 The core of pmreorder is based on user-provided named markers. Sec‐
151 tions of code can be `marked' depending on their importance, and the
152 degree of reordering can be customized by the use of various provided
153 engines.
154 For this purpose, Valgrind's pmemcheck tool exposes a generic marker
156 macro:
157 · VALGRIND_EMIT_LOG(value)
159 It emits log to store_log during pmemcheck processing. value is a us‐
161 er-defined marker name. For more details about pmemcheck execution see
162 PMEMCHECK STORE LOG section below.
163 Example:
165 main.c
167 .
168 .
169 .
170 VALGRIND_EMIT_LOG("PMREORDER_MEMSET_PERSIST.BEGIN");
171 pmem_memset_persist(...);
173 VALGRIND_EMIT_LOG("PMREORDER_MEMSET_PERSIST.END");
175 .
176 .
177 .
178 There are a few rules for macros creation:
180 · Valid macro can have any name, but begin and end section have to
182 match - they are case sensitive.
183 · Macro must have .BEGIN or .END suffix.
185 · Macros can't be crossed.
187 Defined markers can be assigned engines types and configured through
189 the pmreorder tool using the -x parameter.
190 There are two ways to set macro options:
192 · Using command line interface in format:
194 + Using configuration file in .json format:
196 { “PMREORDER_MARKER_NAME1”=“ReorderName1”, “PMREORDER_MARK‐
198 ER_NAME2”=“ReorderName2” }
199 For more details about available
201 engines types, see ENGINES section above.
202 **libpmemobj**(7) also provides set of macros that allows change
204 reordering engine on library or function level:
205 `<library_name|api_function_name>`
207 Example of configuration on function level:
209 { “pmemobj_open”=“NoReorderNoCheck”, “pmemobj_memcpy_persist”=“Reorder‐
211 Partial” }
212 Example of configuration on library level
214 (affecting all library functions):
215 { “libpmemobj”=“NoReorderNoCheck” }
217 List of marked **libpmemobj**(7) API functions:
219 pmemobj_alloc pmemobj_cancel pmemobj_check pmemobj_close pmemobj_create
221 pmemobj_ctl_exec pmemobj_ctl_set pmemobj_free pmemobj_list_insert pmem‐
222 obj_list_insert_new pmemobj_list_move pmemobj_list_remove pmemobj_mem‐
223 cpy pmemobj_memmove pmemobj_memset pmemobj_memcpy_persist pmemobj_mem‐
224 set_persist pmemobj_open pmemobj_publish pmemobj_realloc pmemobj_re‐
225 serve pmemobj_root pmemobj_root_construct pmemobj_strdup pmemo‐
226 bj_tx_abort pmemobj_tx_add_range pmemobj_tx_add_range_direct pmemo‐
227 bj_tx_alloc pmemobj_tx_commit pmemobj_tx_free pmemobj_tx_publish pmemo‐
228 bj_tx_realloc pmemobj_tx_strdup pmemobj_tx_wcsdup pmemobj_tx_xadd_range
229 pmemobj_tx_xadd_range_direct pmemobj_tx_xalloc pmemobj_tx_zalloc pmemo‐
230 bj_tx_zrealloc pmemobj_wcsdup pmemobj_xalloc pmemobj_xreserve pmemo‐
231 bj_zalloc pmemobj_zrealloc
232 # PMEMCHECK STORE LOG #
234 To generate *store_log* for **pmreorder** run pmemcheck
236 with additional parameters:
237 valgrind
239 –tool=pmemcheck
240 -q
241 –log-stores=yes
242 –print-summary=no
243 –log-file=store_log.log
244 –log-stores-stacktraces=yes
245 –log-stores-stacktraces-depth=2
246 –expect-fence-after-clflush=yes
247 test_binary writer_parameter
248 For further details of pmemcheck parameters see [pmemcheck documentation](https://github.com/pmem/valgrind/blob/pmem-3.13/pmemcheck/docs/pmc-manual.xml)
250 # ENVIRONMENT #
253 By default all logging from PMDK libraries is disabled.
255 To enable API macros logging set environment variable:
256 + **PMREORDER_EMIT_LOG**=1
258 # EXAMPLE #
261 python pmreorder.py
263 -l store_log.log
264 -r NoReorderDoCheck
265 -o pmreorder_out.log
266 -c prog
267 -x PMREORDER_MARKER_NAME=ReorderPartial
268 -p checker_binary checker_parameter ```
269 Checker binary will be used to run consistency checks on
271 “store_log.log”, output of pmemcheck tool. Any inconsistent stores
272 found during pmreorder analysis will be logged to pmreorder_out.log.
273
275 <http://pmem.io>
276PMDK - pmreorder version 1.5 2018-10-12 PMREORDER(1)