1CTDB-STATISTICS(7) CTDB - clustered TDB database CTDB-STATISTICS(7)
2
6 ctdb-statistics - CTDB statistics output
7
9 CTDB maintains information about various messages communicated and some
10 of the important operations per node. See the ctdb(1) commands
11 statistics and statisticsreset for displaying statistics.
12 Example: ctdb statistics
14 CTDB version 1
15 Current time of statistics : Fri Sep 12 13:32:32 2014
16 Statistics collected since : (000 01:49:20) Fri Sep 12 11:43:12 2014
17 num_clients 6
18 frozen 0
19 recovering 0
20 num_recoveries 2
21 client_packets_sent 281293
22 client_packets_recv 296317
23 node_packets_sent 452387
24 node_packets_recv 182394
25 keepalive_packets_sent 3927
26 keepalive_packets_recv 3928
27 node
28 req_call 48605
29 reply_call 1
30 req_dmaster 23404
31 reply_dmaster 24917
32 reply_error 0
33 req_message 958
34 req_control 197513
35 reply_control 153705
36 client
37 req_call 130866
38 req_message 770
39 req_control 168921
40 timeouts
41 call 0
42 control 0
43 traverse 0
44 locks
45 num_calls 220
46 num_current 0
47 num_pending 0
48 num_failed 0
49 total_calls 130866
50 pending_calls 0
51 childwrite_calls 1
52 pending_childwrite_calls 0
53 memory_used 334490
54 max_hop_count 18
55 total_ro_delegations 2
56 total_ro_revokes 2
57 hop_count_buckets: 42816 5464 26 1 0 0 0 0 0 0 0 0 0 0 0 0
58 lock_buckets: 9 165 14 15 7 2 2 0 0 0 0 0 0 0 0 0
59 locks_latency MIN/AVG/MAX 0.000685/0.160302/6.369342 sec out of 214
60 reclock_ctdbd MIN/AVG/MAX 0.004940/0.004969/0.004998 sec out of 2
61 reclock_recd MIN/AVG/MAX 0.000000/0.000000/0.000000 sec out of 0
62 call_latency MIN/AVG/MAX 0.000006/0.000719/4.562991 sec out of 126626
63 childwrite_latency MIN/AVG/MAX 0.014527/0.014527/0.014527 sec out of 1
64 CTDB version
67 Version of the ctdb protocol used by the node.
68 Current time of statistics
70 Time when the statistics are generated.
71 This is useful when collecting statistics output periodically for
73 post-processing.
74 Statistics collected since
76 Time when ctdb was started or the last time statistics was reset. The
77 output shows the duration and the timestamp.
78 num_clients
80 Number of processes currently connected to CTDB's unix socket. This
81 includes recovery daemon, ctdb tool and samba processes (smbd,
82 winbindd).
83 frozen
85 1 if the databases are currently frozen, 0 otherwise.
86 recovering
88 1 if recovery is active, 0 otherwise.
89 num_recoveries
91 Number of recoveries since the start of ctdb or since the last
92 statistics reset.
93 client_packets_sent
95 Number of packets sent to client processes via unix domain socket.
96 client_packets_recv
98 Number of packets received from client processes via unix domain
99 socket.
100 node_packets_sent
102 Number of packets sent to the other nodes in the cluster via TCP.
103 node_packets_recv
105 Number of packets received from the other nodes in the cluster via TCP.
106 keepalive_packets_sent
108 Number of keepalive messages sent to other nodes.
109 CTDB periodically sends keepalive messages to other nodes. See
111 KeepaliveInterval tunable in ctdb-tunables(7) for more details.
112 keepalive_packets_recv
114 Number of keepalive messages received from other nodes.
115 node
117 This section lists various types of messages processed which originated
118 from other nodes via TCP.
119 req_call
121 Number of REQ_CALL messages from the other nodes.
122 reply_call
124 Number of REPLY_CALL messages from the other nodes.
125 req_dmaster
127 Number of REQ_DMASTER messages from the other nodes.
128 reply_dmaster
130 Number of REPLY_DMASTER messages from the other nodes.
131 reply_error
133 Number of REPLY_ERROR messages from the other nodes.
134 req_message
136 Number of REQ_MESSAGE messages from the other nodes.
137 req_control
139 Number of REQ_CONTROL messages from the other nodes.
140 reply_control
142 Number of REPLY_CONTROL messages from the other nodes.
143 req_tunnel
145 Number of REQ_TUNNEL messages from the other nodes.
146 client
148 This section lists various types of messages processed which originated
149 from clients via unix domain socket.
150 req_call
152 Number of REQ_CALL messages from the clients.
153 req_message
155 Number of REQ_MESSAGE messages from the clients.
156 req_control
158 Number of REQ_CONTROL messages from the clients.
159 req_tunnel
161 Number of REQ_TUNNEL messages from the clients.
162 timeouts
164 This section lists timeouts occurred when sending various messages.
165 call
167 Number of timeouts for REQ_CALL messages.
168 control
170 Number of timeouts for REQ_CONTROL messages.
171 traverse
173 Number of timeouts for database traverse operations.
174 locks
176 This section lists locking statistics.
177 num_calls
179 Number of completed lock calls. This includes database locks and
180 record locks.
181 num_current
183 Number of scheduled lock calls. This includes database locks and
184 record locks.
185 num_pending
187 Number of queued lock calls. This includes database locks and
188 record locks.
189 num_failed
191 Number of failed lock calls. This includes database locks and
192 record locks.
193 total_calls
195 Number of req_call messages processed from clients. This number should
196 be same as client --> req_call.
197 pending_calls
199 Number of req_call messages which are currently being processed. This
200 number indicates the number of record migrations in flight.
201 childwrite_calls
203 Number of record update calls. Record update calls are used to update a
204 record under a transaction.
205 pending_childwrite_calls
207 Number of record update calls currently active.
208 memory_used
210 The amount of memory in bytes currently used by CTDB using talloc. This
211 includes all the memory used for CTDB's internal data structures. This
212 does not include the memory mapped TDB databases.
213 max_hop_count
215 The maximum number of hops required for a record migration request to
216 obtain the record. High numbers indicate record contention.
217 total_ro_delegations
219 Number of readonly delegations created.
220 total_ro_revokes
222 Number of readonly delegations that were revoked. The difference
223 between total_ro_revokes and total_ro_delegations gives the number of
224 currently active readonly delegations.
225 hop_count_buckets
227 Distribution of migration requests based on hop counts values. Buckets
228 are 0, < 2, < 4, < 8, < 16, < 32, < 64, < 128, < 256, < 512, < 1024,
229 < 2048, < 4096, < 8192, < 16384, ≥ 16384.
230 lock_buckets
232 Distribution of record lock requests based on time required to obtain
233 locks. Buckets are < 1ms, < 10ms, < 100ms, < 1s, < 2s, < 4s, < 8s,
234 < 16s, < 32s, < 64s, ≥ 64s.
235 locks_latency
237 The minimum, the average and the maximum time (in seconds) required to
238 obtain record locks.
239 reclock_ctdbd
241 The minimum, the average and the maximum time (in seconds) required to
242 check if recovery lock is still held by recovery daemon when recovery
243 mode is changed. This check is done in ctdb daemon.
244 reclock_recd
246 The minimum, the average and the maximum time (in seconds) required to
247 check if recovery lock is still held by recovery daemon during
248 recovery. This check is done in recovery daemon.
249 call_latency
251 The minimum, the average and the maximum time (in seconds) required to
252 process a REQ_CALL message from client. This includes the time required
253 to migrate a record from remote node, if the record is not available on
254 the local node.
255 childwrite_latency
257 Default: 0
258 The minimum, the average and the maximum time (in seconds) required to
260 update records under a transaction.
261
263 CTDB maintains per database statistics about important operations. See
264 the ctdb(1) command dbstatistics for displaying database statistics.
265 Example: ctdb dbstatistics notify_index.tdb
267 DB Statistics: notify_index.tdb
268 ro_delegations 0
269 ro_revokes 0
270 locks
271 total 131
272 failed 0
273 current 0
274 pending 0
275 hop_count_buckets: 9890 5454 26 1 0 0 0 0 0 0 0 0 0 0 0 0
276 lock_buckets: 4 117 10 0 0 0 0 0 0 0 0 0 0 0 0 0
277 locks_latency MIN/AVG/MAX 0.000683/0.004198/0.014730 sec out of 131
278 Num Hot Keys: 3
279 Count:7 Key:2f636c75737465726673
280 Count:18 Key:2f636c757374657266732f64617461
281 Count:7 Key:2f636c757374657266732f646174612f636c69656e7473
282 DB Statistics
285 Name of the database.
286 ro_delegations
288 Number of readonly delegations created in the database.
289 ro_revokes
291 Number of readonly delegations revoked. The difference in
292 ro_delegations and ro_revokes indicates the currently active readonly
293 delegations.
294 locks
296 This section lists locking statistics.
297 total
299 Number of completed lock calls. This includes database locks and
300 record locks.
301 failed
303 Number of failed lock calls. This includes database locks and
304 record locks.
305 current
307 Number of scheduled lock calls. This includes database locks and
308 record locks.
309 pending
311 Number of queued lock calls. This includes database locks and
312 record locks.
313 hop_count_buckets
315 Distribution of migration requests based on hop counts values. Buckets
316 are 0, < 2, < 4, < 8, < 16, < 32, < 64, < 128, < 256, < 512, < 1024,
317 < 2048, < 4096, < 8192, < 16384, ≥ 16384.
318 lock_buckets
320 Distribution of record lock requests based on time required to obtain
321 locks. Buckets are < 1ms, < 10ms, < 100ms, < 1s, < 2s, < 4s, < 8s,
322 < 16s, < 32s, < 64s, ≥ 64s.
323 locks_latency
325 The minimum, the average and the maximum time (in seconds) required to
326 obtain record locks.
327 Num Hot Keys
329 Number of contended records determined by hop count. CTDB keeps track
330 of top 10 hot records and the output shows hex encoded keys for the hot
331 records.
332
334 ctdb(1), ctdbd(1), ctdb-tunables(7), http://ctdb.samba.org/
335
337 This documentation was written by Amitay Isaacs, Martin Schwenke
338
340 Copyright © 2007 Andrew Tridgell, Ronnie Sahlberg
341 This program is free software; you can redistribute it and/or modify it
343 under the terms of the GNU General Public License as published by the
344 Free Software Foundation; either version 3 of the License, or (at your
345 option) any later version.
346 This program is distributed in the hope that it will be useful, but
348 WITHOUT ANY WARRANTY; without even the implied warranty of
349 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
350 General Public License for more details.
351 You should have received a copy of the GNU General Public License along
353 with this program; if not, see http://www.gnu.org/licenses.
354ctdb 03/25/2021 CTDB-STATISTICS(7)