1disk_log(3) Erlang Module Definition disk_log(3)
2
6 disk_log - A disk-based term logging facility.
7
9 disk_log is a disk-based term logger that enables efficient logging of
10 items on files.
11 Two types of logs are supported:
13 halt logs:
15 Appends items to a single file, which size can be limited by the
16 disk_log module.
17 wrap logs:
19 Uses a sequence of wrap log files of limited size. As a wrap log
20 file is filled up, further items are logged on to the next file in
21 the sequence, starting all over with the first file when the last
22 file is filled up.
23 For efficiency reasons, items are always written to files as binaries.
25 Two formats of the log files are supported:
27 internal format:
29 Supports automatic repair of log files that are not properly closed
30 and enables efficient reading of logged items in chunks using a set
31 of functions defined in this module. This is the only way to read
32 internally formatted logs. An item logged to an internally format‐
33 ted log must not occupy more than 4 GB of disk space (the size must
34 fit in 4 bytes).
35 external format:
37 Leaves it up to the user to read and interpret the logged data. The
38 disk_log module cannot repair externally formatted logs.
39 For each open disk log, one process handles requests made to the disk
41 log. This process is created when open/1 is called, provided there
42 exists no process handling the disk log. A process that opens a disk
43 log can be an owner or an anonymous user of the disk log. Each owner is
44 linked to the disk log process, and an owner can close the disk log
45 either explicitly (by calling close/1 or lclose/1,2) or by terminating.
46 Owners can subscribe to notifications, messages of the form {disk_log,
48 Node, Log, Info}, which are sent from the disk log process when certain
49 events occur, see the functions and in particular the open/1 option
50 notify. A log can have many owners, but a process cannot own a log more
51 than once. However, the same process can open the log as a user more
52 than once.
53 For a disk log process to close its file properly and terminate, it
55 must be closed by its owners and once by some non-owner process for
56 each time the log was used anonymously. The users are counted and there
57 must not be any users left when the disk log process terminates.
58 Items can be logged synchronously by using functions log/2, blog/2,
60 log_terms/2, and blog_terms/2. For each of these functions, the caller
61 is put on hold until the items are logged (but not necessarily written,
62 use sync/1 to ensure that). By adding an a to each of the mentioned
63 function names, we get functions that log items asynchronously. Asyn‐
64 chronous functions do not wait for the disk log process to write the
65 items to the file, but return the control to the caller more or less
66 immediately.
67 When using the internal format for logs, use functions log/2,
69 log_terms/2, alog/2, and alog_terms/2. These functions log one or more
70 Erlang terms. By prefixing each of the functions with a b (for
71 "binary"), we get the corresponding blog() functions for the external
72 format. These functions log one or more chunks of bytes. For example,
73 to log the string "hello" in ASCII format, you can use
74 disk_log:blog(Log, "hello"), or disk_log:blog(Log,
75 list_to_binary("hello")). The two alternatives are equally efficient.
76 The blog() functions can also be used for internally formatted logs,
78 but in this case they must be called with binaries constructed with
79 calls to term_to_binary/1. There is no check to ensure this, it is
80 entirely the responsibility of the caller. If these functions are
81 called with binaries that do not correspond to Erlang terms, the
82 chunk/2,3 and automatic repair functions fail. The corresponding terms
83 (not the binaries) are returned when chunk/2,3 is called.
84 A collection of open disk logs with the same name running on different
86 nodes is said to be a distributed disk log if requests made to any of
87 the logs are automatically made to the other logs as well. The members
88 of such a collection are called individual distributed disk logs, or
89 just distributed disk logs if there is no risk of confusion. There is
90 no order between the members of such a collection. For example, logged
91 terms are not necessarily written to the node where the request was
92 made before written to the other nodes. However, a few functions do not
93 make requests to all members of distributed disk logs, namely info/1,
94 chunk/2,3, bchunk/2,3, chunk_step/3, and lclose/1,2.
95 An open disk log that is not a distributed disk log is said to be a
97 local disk log. A local disk log is only accessible from the node where
98 the disk log process runs, whereas a distributed disk log is accessible
99 from all nodes in the Erlang system, except for those nodes where a
100 local disk log with the same name as the distributed disk log exists.
101 All processes on nodes that have access to a local or distributed disk
102 log can log items or otherwise change, inspect, or close the log.
103 It is not guaranteed that all log files of a distributed disk log con‐
105 tain the same log items. No attempt is made to synchronize the contents
106 of the files. However, as long as at least one of the involved nodes is
107 alive at each time, all items are logged. When logging items to a dis‐
108 tributed log, or otherwise trying to change the log, the replies from
109 individual logs are ignored. If all nodes are down, the disk log func‐
110 tions reply with a nonode error.
111 Note:
113 In some applications, it can be unacceptable that replies from individ‐
114 ual logs are ignored. An alternative in such situations is to use many
115 local disk logs instead of one distributed disk log, and implement the
116 distribution without use of the disk_log module.
117 Errors are reported differently for asynchronous log attempts and other
120 uses of the disk_log module. When used synchronously, this module
121 replies with an error message, but when called asynchronously, this
122 module does not know where to send the error message. Instead, owners
123 subscribing to notifications receive an error_status message.
124 The disk_log module does not report errors to the error_logger module.
126 It is up to the caller to decide whether to employ the error logger.
127 Function format_error/1 can be used to produce readable messages from
128 error replies. However, information events are sent to the error logger
129 in two situations, namely when a log is repaired, or when a file is
130 missing while reading chunks.
131 Error message no_such_log means that the specified disk log is not
133 open. Nothing is said about whether the disk log files exist or not.
134 Note:
136 If an attempt to reopen or truncate a log fails (see reopen/2,3 and
137 truncate/1,2) the disk log process terminates immediately. Before the
138 process terminates, links to owners and blocking processes (see
139 block/1,2) are removed. The effect is that the links work in one direc‐
140 tion only. Any process using a disk log must check for error message
141 no_such_log if some other process truncates or reopens the log simulta‐
142 neously.
143
146 log() = term()
147 dlog_size() =
149 infinity |
150 integer() >= 1 |
151 {MaxNoBytes :: integer() >= 1, MaxNoFiles :: integer() >= 1}
152 dlog_format() = external | internal
154 dlog_head_opt() = none | term() | iodata()
156 dlog_mode() = read_only | read_write
158 dlog_type() = halt | wrap
160 continuation()
162 Chunk continuation returned by chunk/2,3, bchunk/2,3, or
164 chunk_step/3.
165 invalid_header() = term()
167 file_error() = term()
169
171 accessible_logs() -> {[LocalLog], [DistributedLog]}
172 Types:
174 LocalLog = DistributedLog = log()
176 Returns the names of the disk logs accessible on the current
178 node. The first list contains local disk logs and the second
179 list contains distributed disk logs.
180 alog(Log, Term) -> notify_ret()
182 balog(Log, Bytes) -> notify_ret()
184 Types:
186 Log = log()
188 Term = term()
189 Bytes = iodata()
190 notify_ret() = ok | {error, no_such_log}
191 Asynchronously append an item to a disk log. alog/2 is used for
193 internally formatted logs and balog/2 for externally formatted
194 logs. balog/2 can also be used for internally formatted logs if
195 the binary is constructed with a call to term_to_binary/1.
196 Owners subscribing to notifications receive message read_only,
198 blocked_log, or format_external if the item cannot be written on
199 the log, and possibly one of the messages wrap, full, or
200 error_status if an item is written on the log. Message
201 error_status is sent if something is wrong with the header func‐
202 tion or if a file error occurs.
203 alog_terms(Log, TermList) -> notify_ret()
205 balog_terms(Log, ByteList) -> notify_ret()
207 Types:
209 Log = log()
211 TermList = [term()]
212 ByteList = [iodata()]
213 notify_ret() = ok | {error, no_such_log}
214 Asynchronously append a list of items to a disk log.
216 alog_terms/2 is used for internally formatted logs and
217 balog_terms/2 for externally formatted logs. balog_terms/2 can
218 also be used for internally formatted logs if the binaries are
219 constructed with calls to term_to_binary/1.
220 Owners subscribing to notifications receive message read_only,
222 blocked_log, or format_external if the items cannot be written
223 on the log, and possibly one or more of the messages wrap, full,
224 and error_status if items are written on the log. Message
225 error_status is sent if something is wrong with the header func‐
226 tion or if a file error occurs.
227 block(Log) -> ok | {error, block_error_rsn()}
229 block(Log, QueueLogRecords) -> ok | {error, block_error_rsn()}
231 Types:
233 Log = log()
235 QueueLogRecords = boolean()
236 block_error_rsn() = no_such_log | nonode | {blocked_log, log()}
237 With a call to block/1,2 a process can block a log. If the
239 blocking process is not an owner of the log, a temporary link is
240 created between the disk log process and the blocking process.
241 The link ensures that the disk log is unblocked if the blocking
242 process terminates without first closing or unblocking the log.
243 Any process can probe a blocked log with info/1 or close it with
245 close/1. The blocking process can also use functions chunk/2,3,
246 bchunk/2,3, chunk_step/3, and unblock/1 without being affected
247 by the block. Any other attempt than those mentioned so far to
248 update or read a blocked log suspends the calling process until
249 the log is unblocked or returns error message {blocked_log,
250 Log}, depending on whether the value of QueueLogRecords is true
251 or false. QueueLogRecords defaults to true, which is used by
252 block/1.
253 change_header(Log, Header) -> ok | {error, Reason}
255 Types:
257 Log = log()
259 Header =
260 {head, dlog_head_opt()} |
261 {head_func, MFA :: {atom(), atom(), list()}}
262 Reason =
263 no_such_log |
264 nonode |
265 {read_only_mode, Log} |
266 {blocked_log, Log} |
267 {badarg, head}
268 Changes the value of option head or head_func for an owner of a
270 disk log.
271 change_notify(Log, Owner, Notify) -> ok | {error, Reason}
273 Types:
275 Log = log()
277 Owner = pid()
278 Notify = boolean()
279 Reason =
280 no_such_log |
281 nonode |
282 {blocked_log, Log} |
283 {badarg, notify} |
284 {not_owner, Owner}
285 Changes the value of option notify for an owner of a disk log.
287 change_size(Log, Size) -> ok | {error, Reason}
289 Types:
291 Log = log()
293 Size = dlog_size()
294 Reason =
295 no_such_log |
296 nonode |
297 {read_only_mode, Log} |
298 {blocked_log, Log} |
299 {new_size_too_small, Log, CurrentSize :: integer() >= 1}
300 |
301 {badarg, size} |
302 {file_error, file:filename(), file_error()}
303 Changes the size of an open log. For a halt log, the size can
305 always be increased, but it cannot be decreased to something
306 less than the current file size.
307 For a wrap log, both the size and the number of files can always
309 be increased, as long as the number of files does not exceed
310 65000. If the maximum number of files is decreased, the change
311 is not valid until the current file is full and the log wraps to
312 the next file. The redundant files are removed the next time the
313 log wraps around, that is, starts to log to file number 1.
314 As an example, assume that the old maximum number of files is 10
316 and that the new maximum number of files is 6. If the current
317 file number is not greater than the new maximum number of files,
318 files 7-10 are removed when file 6 is full and the log starts to
319 write to file number 1 again. Otherwise, the files greater than
320 the current file are removed when the current file is full (for
321 example, if the current file is 8, files 9 and 10 are removed).
322 The files between the new maximum number of files and the cur‐
323 rent file (that is, files 7 and 8) are removed the next time
324 file 6 is full.
325 If the size of the files is decreased, the change immediately
327 affects the current log. It does not change the size of log
328 files already full until the next time they are used.
329 If the log size is decreased, for example, to save space, func‐
331 tion inc_wrap_file/1 can be used to force the log to wrap.
332 chunk(Log, Continuation) -> chunk_ret()
334 chunk(Log, Continuation, N) -> chunk_ret()
336 bchunk(Log, Continuation) -> bchunk_ret()
338 bchunk(Log, Continuation, N) -> bchunk_ret()
340 Types:
342 Log = log()
344 Continuation = start | continuation()
345 N = integer() >= 1 | infinity
346 chunk_ret() =
347 {Continuation2 :: continuation(), Terms :: [term()]} |
348 {Continuation2 :: continuation(),
349 Terms :: [term()],
350 Badbytes :: integer() >= 0} |
351 eof |
352 {error, Reason :: chunk_error_rsn()}
353 bchunk_ret() =
354 {Continuation2 :: continuation(), Binaries :: [binary()]} |
355 {Continuation2 :: continuation(),
356 Binaries :: [binary()],
357 Badbytes :: integer() >= 0} |
358 eof |
359 {error, Reason :: chunk_error_rsn()}
360 chunk_error_rsn() =
361 no_such_log |
362 {format_external, log()} |
363 {blocked_log, log()} |
364 {badarg, continuation} |
365 {not_internal_wrap, log()} |
366 {corrupt_log_file, FileName :: file:filename()} |
367 {file_error, file:filename(), file_error()}
368 Efficiently reads the terms that are appended to an internally
370 formatted log. It minimizes disk I/O by reading 64 kilobyte
371 chunks from the file. Functions bchunk/2,3 return the binaries
372 read from the file, they do not call binary_to_term(). Apart
373 from that, they work just like chunk/2,3.
374 The first time chunk() (or bchunk()) is called, an initial con‐
376 tinuation, the atom start, must be provided. If a disk log
377 process is running on the current node, terms are read from that
378 log. Otherwise, an individual distributed log on some other node
379 is chosen, if such a log exists.
380 When chunk/3 is called, N controls the maximum number of terms
382 that are read from the log in each chunk. Defaults to infinity,
383 which means that all the terms contained in the 64 kilobyte
384 chunk are read. If less than N terms are returned, this does not
385 necessarily mean that the end of the file is reached.
386 chunk() returns a tuple {Continuation2, Terms}, where Terms is a
388 list of terms found in the log. Continuation2 is yet another
389 continuation, which must be passed on to any subsequent calls to
390 chunk(). With a series of calls to chunk(), all terms from a log
391 can be extracted.
392 chunk() returns a tuple {Continuation2, Terms, Badbytes} if the
394 log is opened in read-only mode and the read chunk is corrupt.
395 Badbytes is the number of bytes in the file found not to be
396 Erlang terms in the chunk. Notice that the log is not repaired.
397 When trying to read chunks from a log opened in read-write mode,
398 tuple {corrupt_log_file, FileName} is returned if the read chunk
399 is corrupt.
400 chunk() returns eof when the end of the log is reached, or
402 {error, Reason} if an error occurs. If a wrap log file is miss‐
403 ing, a message is output on the error log.
404 When chunk/2,3 is used with wrap logs, the returned continuation
406 might not be valid in the next call to chunk(). This is because
407 the log can wrap and delete the file into which the continuation
408 points. To prevent this, the log can be blocked during the
409 search.
410 chunk_info(Continuation) -> InfoList | {error, Reason}
412 Types:
414 Continuation = continuation()
416 InfoList = [{node, Node :: node()}, ...]
417 Reason = {no_continuation, Continuation}
418 Returns the pair {node, Node}, describing the chunk continuation
420 returned by chunk/2,3, bchunk/2,3, or chunk_step/3.
421 Terms are read from the disk log running on Node.
423 chunk_step(Log, Continuation, Step) ->
425 {ok, any()} | {error, Reason}
426 Types:
428 Log = log()
430 Continuation = start | continuation()
431 Step = integer()
432 Reason =
433 no_such_log |
434 end_of_log |
435 {format_external, Log} |
436 {blocked_log, Log} |
437 {badarg, continuation} |
438 {file_error, file:filename(), file_error()}
439 Can be used with chunk/2,3 and bchunk/2,3 to search through an
441 internally formatted wrap log. It takes as argument a continua‐
442 tion as returned by chunk/2,3, bchunk/2,3, or chunk_step/3, and
443 steps forward (or backward) Step files in the wrap log. The con‐
444 tinuation returned, points to the first log item in the new cur‐
445 rent file.
446 If atom start is specified as continuation, a disk log to read
448 terms from is chosen. A local or distributed disk log on the
449 current node is preferred to an individual distributed log on
450 some other node.
451 If the wrap log is not full because all files are not yet used,
453 {error, end_of_log} is returned if trying to step outside the
454 log.
455 close(Log) -> ok | {error, close_error_rsn()}
457 Types:
459 Log = log()
461 close_error_rsn() =
462 no_such_log |
463 nonode |
464 {file_error, file:filename(), file_error()}
465 Closes a local or distributed disk log properly. An internally
467 formatted log must be closed before the Erlang system is
468 stopped. Otherwise, the log is regarded as unclosed and the
469 automatic repair procedure is activated next time the log is
470 opened.
471 The disk log process is not terminated as long as there are own‐
473 ers or users of the log. All owners must close the log, possibly
474 by terminating. Also, any other process, not only the processes
475 that have opened the log anonymously, can decrement the users
476 counter by closing the log. Attempts to close a log by a process
477 that is not an owner are ignored if there are no users.
478 If the log is blocked by the closing process, the log is also
480 unblocked.
481 format_error(Error) -> io_lib:chars()
483 Types:
485 Error = term()
487 Given the error returned by any function in this module, this
489 function returns a descriptive string of the error in English.
490 For file errors, function format_error/1 in module file is
491 called.
492 inc_wrap_file(Log) -> ok | {error, inc_wrap_error_rsn()}
494 Types:
496 Log = log()
498 inc_wrap_error_rsn() =
499 no_such_log |
500 nonode |
501 {read_only_mode, log()} |
502 {blocked_log, log()} |
503 {halt_log, log()} |
504 {invalid_header, invalid_header()} |
505 {file_error, file:filename(), file_error()}
506 invalid_header() = term()
507 Forces the internally formatted disk log to start logging to the
509 next log file. It can be used, for example, with change_size/2
510 to reduce the amount of disk space allocated by the disk log.
511 Owners subscribing to notifications normally receive a wrap mes‐
513 sage, but if an error occurs with a reason tag of invalid_header
514 or file_error, an error_status message is sent.
515 info(Log) -> InfoList | {error, no_such_log}
517 Types:
519 Log = log()
521 InfoList = [dlog_info()]
522 dlog_info() =
523 {name, Log :: log()} |
524 {file, File :: file:filename()} |
525 {type, Type :: dlog_type()} |
526 {format, Format :: dlog_format()} |
527 {size, Size :: dlog_size()} |
528 {mode, Mode :: dlog_mode()} |
529 {owners, [{pid(), Notify :: boolean()}]} |
530 {users, Users :: integer() >= 0} |
531 {status,
532 Status :: ok | {blocked, QueueLogRecords :: boolean()}} |
533 {node, Node :: node()} |
534 {distributed, Dist :: local | [node()]} |
535 {head,
536 Head ::
537 none | {head, term()} | (MFA :: {atom(), atom(), list()})} |
538 {no_written_items, NoWrittenItems :: integer() >= 0} |
539 {full, Full :: boolean} |
540 {no_current_bytes, integer() >= 0} |
541 {no_current_items, integer() >= 0} |
542 {no_items, integer() >= 0} |
543 {current_file, integer() >= 1} |
544 {no_overflows,
545 {SinceLogWasOpened :: integer() >= 0,
546 SinceLastInfo :: integer() >= 0}}
547 Returns a list of {Tag, Value} pairs describing the log. If a
549 disk log process is running on the current node, that log is
550 used as source of information, otherwise an individual distrib‐
551 uted log on some other node is chosen, if such a log exists.
552 The following pairs are returned for all logs:
554 {name, Log}:
556 Log is the log name as specified by the open/1 option name.
557 {file, File}:
559 For halt logs File is the filename, and for wrap logs File
560 is the base name.
561 {type, Type}:
563 Type is the log type as specified by the open/1 option type.
564 {format, Format}:
566 Format is the log format as specified by the open/1 option
567 format.
568 {size, Size}:
570 Size is the log size as specified by the open/1 option size,
571 or the size set by change_size/2. The value set by
572 change_size/2 is reflected immediately.
573 {mode, Mode}:
575 Mode is the log mode as specified by the open/1 option mode.
576 {owners, [{pid(), Notify}]}:
578 Notify is the value set by the open/1 option notify or func‐
579 tion change_notify/3 for the owners of the log.
580 {users, Users}:
582 Users is the number of anonymous users of the log, see the
583 open/1 option linkto.
584 {status, Status}:
586 Status is ok or {blocked, QueueLogRecords} as set by func‐
587 tions block/1,2 and unblock/1.
588 {node, Node}:
590 The information returned by the current invocation of func‐
591 tion info/1 is gathered from the disk log process running on
592 Node.
593 {distributed, Dist}:
595 If the log is local on the current node, Dist has the value
596 local, otherwise all nodes where the log is distributed are
597 returned as a list.
598 The following pairs are returned for all logs opened in
600 read_write mode:
601 {head, Head}:
603 Depending on the value of the open/1 options head and
604 head_func, or set by function change_header/2, the value of
605 Head is none (default), {head, H} (head option), or {M,F,A}
606 (head_func option).
607 {no_written_items, NoWrittenItems}:
609 NoWrittenItems is the number of items written to the log
610 since the disk log process was created.
611 The following pair is returned for halt logs opened in
613 read_write mode:
614 {full, Full}:
616 Full is true or false depending on whether the halt log is
617 full or not.
618 The following pairs are returned for wrap logs opened in
620 read_write mode:
621 {no_current_bytes, integer() >= 0}:
623 The number of bytes written to the current wrap log file.
624 {no_current_items, integer() >= 0}:
626 The number of items written to the current wrap log file,
627 header inclusive.
628 {no_items, integer() >= 0}:
630 The total number of items in all wrap log files.
631 {current_file, integer()}:
633 The ordinal for the current wrap log file in the range
634 1..MaxNoFiles, where MaxNoFiles is specified by the open/1
635 option size or set by change_size/2.
636 {no_overflows, {SinceLogWasOpened, SinceLastInfo}}:
638 SinceLogWasOpened (SinceLastInfo) is the number of times a
639 wrap log file has been filled up and a new one is opened or
640 inc_wrap_file/1 has been called since the disk log was last
641 opened (info/1 was last called). The first time info/2 is
642 called after a log was (re)opened or truncated, the two val‐
643 ues are equal.
644 Notice that functions chunk/2,3, bchunk/2,3, and chunk_step/3 do
646 not affect any value returned by info/1.
647 lclose(Log) -> ok | {error, lclose_error_rsn()}
649 lclose(Log, Node) -> ok | {error, lclose_error_rsn()}
651 Types:
653 Log = log()
655 Node = node()
656 lclose_error_rsn() =
657 no_such_log | {file_error, file:filename(), file_error()}
658 lclose/1 closes a local log or an individual distributed log on
660 the current node.
661 lclose/2 closes an individual distributed log on the specified
663 node if the node is not the current one.
664 lclose(Log) is equivalent to lclose(Log, node()). See also
666 close/1.
667 If no log with the specified name exist on the specified node,
669 no_such_log is returned.
670 log(Log, Term) -> ok | {error, Reason :: log_error_rsn()}
672 blog(Log, Bytes) -> ok | {error, Reason :: log_error_rsn()}
674 Types:
676 Log = log()
678 Term = term()
679 Bytes = iodata()
680 log_error_rsn() =
681 no_such_log |
682 nonode |
683 {read_only_mode, log()} |
684 {format_external, log()} |
685 {blocked_log, log()} |
686 {full, log()} |
687 {invalid_header, invalid_header()} |
688 {file_error, file:filename(), file_error()}
689 Synchronously appends a term to a disk log. Returns ok or
691 {error, Reason} when the term is written to disk. If the log is
692 distributed, ok is returned, unless all nodes are down. Terms
693 are written by the ordinary write() function of the operating
694 system. Hence, it is not guaranteed that the term is written to
695 disk, it can linger in the operating system kernel for a while.
696 To ensure that the item is written to disk, function sync/1 must
697 be called.
698 log/2 is used for internally formatted logs, and blog/2 for
700 externally formatted logs. blog/2 can also be used for inter‐
701 nally formatted logs if the binary is constructed with a call to
702 term_to_binary/1.
703 Owners subscribing to notifications are notified of an error
705 with an error_status message if the error reason tag is
706 invalid_header or file_error.
707 log_terms(Log, TermList) ->
709 ok | {error, Resaon :: log_error_rsn()}
710 blog_terms(Log, BytesList) ->
712 ok | {error, Reason :: log_error_rsn()}
713 Types:
715 Log = log()
717 TermList = [term()]
718 BytesList = [iodata()]
719 log_error_rsn() =
720 no_such_log |
721 nonode |
722 {read_only_mode, log()} |
723 {format_external, log()} |
724 {blocked_log, log()} |
725 {full, log()} |
726 {invalid_header, invalid_header()} |
727 {file_error, file:filename(), file_error()}
728 Synchronously appends a list of items to the log. It is more
730 efficient to use these functions instead of functions log/2 and
731 blog/2. The specified list is split into as large sublists as
732 possible (limited by the size of wrap log files), and each sub‐
733 list is logged as one single item, which reduces the overhead.
734 log_terms/2 is used for internally formatted logs, and
736 blog_terms/2 for externally formatted logs. blog_terms/2 can
737 also be used for internally formatted logs if the binaries are
738 constructed with calls to term_to_binary/1.
739 Owners subscribing to notifications are notified of an error
741 with an error_status message if the error reason tag is
742 invalid_header or file_error.
743 open(ArgL) -> open_ret() | dist_open_ret()
745 Types:
747 ArgL = dlog_options()
749 dlog_options() = [dlog_option()]
750 dlog_option() =
751 {name, Log :: log()} |
752 {file, FileName :: file:filename()} |
753 {linkto, LinkTo :: none | pid()} |
754 {repair, Repair :: true | false | truncate} |
755 {type, Type :: dlog_type()} |
756 {format, Format :: dlog_format()} |
757 {size, Size :: dlog_size()} |
758 {distributed, Nodes :: [node()]} |
759 {notify, boolean()} |
760 {head, Head :: dlog_head_opt()} |
761 {head_func, MFA :: {atom(), atom(), list()}} |
762 {quiet, boolean()} |
763 {mode, Mode :: dlog_mode()}
764 open_ret() = ret() | {error, open_error_rsn()}
765 ret() =
766 {ok, Log :: log()} |
767 {repaired,
768 Log :: log(),
769 {recovered, Rec :: integer() >= 0},
770 {badbytes, Bad :: integer() >= 0}}
771 dist_open_ret() =
772 {[{node(), ret()}], [{node(), {error, dist_error_rsn()}}]}
773 dist_error_rsn() = nodedown | open_error_rsn()
774 open_error_rsn() =
775 no_such_log |
776 {badarg, term()} |
777 {size_mismatch,
778 CurrentSize :: dlog_size(),
779 NewSize :: dlog_size()} |
780 {arg_mismatch,
781 OptionName :: dlog_optattr(),
782 CurrentValue :: term(),
783 Value :: term()} |
784 {name_already_open, Log :: log()} |
785 {open_read_write, Log :: log()} |
786 {open_read_only, Log :: log()} |
787 {need_repair, Log :: log()} |
788 {not_a_log_file, FileName :: file:filename()} |
789 {invalid_index_file, FileName :: file:filename()} |
790 {invalid_header, invalid_header()} |
791 {file_error, file:filename(), file_error()} |
792 {node_already_open, Log :: log()}
793 dlog_optattr() =
794 name |
795 file |
796 linkto |
797 repair |
798 type |
799 format |
800 size |
801 distributed |
802 notify |
803 head |
804 head_func |
805 mode
806 dlog_size() =
807 infinity |
808 integer() >= 1 |
809 {MaxNoBytes :: integer() >= 1, MaxNoFiles :: integer() >= 1}
810 Parameter ArgL is a list of the following options:
812 {name, Log}:
814 Specifies the log name. This name must be passed on as a
815 parameter in all subsequent logging operations. A name must
816 always be supplied.
817 {file, FileName}:
819 Specifies the name of the file to be used for logged terms.
820 If this value is omitted and the log name is an atom or a
821 string, the filename defaults to lists:concat([Log, ".LOG"])
822 for halt logs.
823 For wrap logs, this is the base name of the files. Each file
825 in a wrap log is called <base_name>.N, where N is an inte‐
826 ger. Each wrap log also has two files called <base_name>.idx
827 and <base_name>.siz.
828 {linkto, LinkTo}:
830 If LinkTo is a pid, it becomes an owner of the log. If
831 LinkTo is none, the log records that it is used anonymously
832 by some process by incrementing the users counter. By
833 default, the process that calls open/1 owns the log.
834 {repair, Repair}:
836 If Repair is true, the current log file is repaired, if
837 needed. As the restoration is initiated, a message is output
838 on the error log. If false is specified, no automatic repair
839 is attempted. Instead, the tuple {error, {need_repair, Log}}
840 is returned if an attempt is made to open a corrupt log
841 file. If truncate is specified, the log file becomes trun‐
842 cated, creating an empty log. Defaults to true, which has no
843 effect on logs opened in read-only mode.
844 {type, Type}:
846 The log type. Defaults to halt.
847 {format, Format}:
849 Disk log format. Defaults to internal.
850 {size, Size}:
852 Log size.
853 When a halt log has reached its maximum size, all attempts
855 to log more items are rejected. Defaults to infinity, which
856 for halt implies that there is no maximum size.
857 For wrap logs, parameter Size can be a pair {MaxNoBytes,
859 MaxNoFiles} or infinity. In the latter case, if the files of
860 an existing wrap log with the same name can be found, the
861 size is read from the existing wrap log, otherwise an error
862 is returned.
863 Wrap logs write at most MaxNoBytes bytes on each file and
865 use MaxNoFiles files before starting all over with the first
866 wrap log file. Regardless of MaxNoBytes, at least the header
867 (if there is one) and one item are written on each wrap log
868 file before wrapping to the next file.
869 When opening an existing wrap log, it is not necessary to
871 supply a value for option Size, but any supplied value must
872 equal the current log size, otherwise the tuple {error,
873 {size_mismatch, CurrentSize, NewSize}} is returned.
874 {distributed, Nodes}:
876 This option can be used for adding members to a distributed
877 disk log. Defaults to [], which means that the log is local
878 on the current node.
879 {notify, boolean()}:
881 If true, the log owners are notified when certain log events
882 occur. Defaults to false. The owners are sent one of the
883 following messages when an event occurs:
884 {disk_log, Node, Log, {wrap, NoLostItems}}:
886 Sent when a wrap log has filled up one of its files and a
887 new file is opened. NoLostItems is the number of previ‐
888 ously logged items that were lost when truncating existing
889 files.
890 {disk_log, Node, Log, {truncated, NoLostItems}}:
892 Sent when a log is truncated or reopened. For halt logs
893 NoLostItems is the number of items written on the log
894 since the disk log process was created. For wrap logs
895 NoLostItems is the number of items on all wrap log files.
896 {disk_log, Node, Log, {read_only, Items}}:
898 Sent when an asynchronous log attempt is made to a log
899 file opened in read-only mode. Items is the items from the
900 log attempt.
901 {disk_log, Node, Log, {blocked_log, Items}}:
903 Sent when an asynchronous log attempt is made to a blocked
904 log that does not queue log attempts. Items is the items
905 from the log attempt.
906 {disk_log, Node, Log, {format_external, Items}}:
908 Sent when function alog/2 or alog_terms/2 is used for
909 internally formatted logs. Items is the items from the log
910 attempt.
911 {disk_log, Node, Log, full}:
913 Sent when an attempt to log items to a wrap log would
914 write more bytes than the limit set by option size.
915 {disk_log, Node, Log, {error_status, Status}}:
917 Sent when the error status changes. The error status is
918 defined by the outcome of the last attempt to log items to
919 the log, or to truncate the log, or the last use of func‐
920 tion sync/1, inc_wrap_file/1, or change_size/2. Status is
921 either ok or {error, Error}, the former is the initial
922 value.
923 {head, Head}:
925 Specifies a header to be written first on the log file. If
926 the log is a wrap log, the item Head is written first in
927 each new file. Head is to be a term if the format is inter‐
928 nal, otherwise a sequence of bytes. Defaults to none, which
929 means that no header is written first on the file.
930 {head_func, {M,F,A}}:
932 Specifies a function to be called each time a new log file
933 is opened. The call M:F(A) is assumed to return {ok, Head}.
934 The item Head is written first in each file. Head is to be a
935 term if the format is internal, otherwise a sequence of
936 bytes.
937 {mode, Mode}:
939 Specifies if the log is to be opened in read-only or read-
940 write mode. Defaults to read_write.
941 {quiet, Boolean}:
943 Specifies if messages will be sent to error_logger on recov‐
944 erable errors with the log files. Defaults to false.
945 open/1 returns {ok, Log} if the log file is successfully opened.
947 If the file is successfully repaired, the tuple {repaired, Log,
948 {recovered, Rec}, {badbytes, Bad}} is returned, where Rec is the
949 number of whole Erlang terms found in the file and Bad is the
950 number of bytes in the file that are non-Erlang terms. If the
951 parameter distributed is specified, open/1 returns a list of
952 successful replies and a list of erroneous replies. Each reply
953 is tagged with the node name.
954 When a disk log is opened in read-write mode, any existing log
956 file is checked for. If there is none, a new empty log is cre‐
957 ated, otherwise the existing file is opened at the position
958 after the last logged item, and the logging of items starts from
959 there. If the format is internal and the existing file is not
960 recognized as an internally formatted log, a tuple {error,
961 {not_a_log_file, FileName}} is returned.
962 open/1 cannot be used for changing the values of options of an
964 open log. When there are prior owners or users of a log, all
965 option values except name, linkto, and notify are only checked
966 against the values supplied before as option values to function
967 open/1, change_header/2, change_notify/3, or change_size/2.
968 Thus, none of the options except name is mandatory. If some
969 specified value differs from the current value, a tuple {error,
970 {arg_mismatch, OptionName, CurrentValue, Value}} is returned.
971 Note:
973 If an owner attempts to open a log as owner once again, it is
974 acknowledged with the return value {ok, Log}, but the state of
975 the disk log is not affected.
976 If a log with a specified name is local on some node, and one
979 tries to open the log distributed on the same node, the tuple
980 {error, {node_already_open, Log}} is returned. The same tuple is
981 returned if the log is distributed on some node, and one tries
982 to open the log locally on the same node. Opening individual
983 distributed disk logs for the first time adds those logs to a
984 (possibly empty) distributed disk log. The supplied option val‐
985 ues are used on all nodes mentioned by option distributed. Indi‐
986 vidual distributed logs know nothing about each other's option
987 values, so each node can be given unique option values by creat‐
988 ing a distributed log with many calls to open/1.
989 A log file can be opened more than once by giving different val‐
991 ues to option name or by using the same file when distributing a
992 log on different nodes. It is up to the user of module disk_log
993 to ensure that not more than one disk log process has write
994 access to any file, otherwise the file can be corrupted.
995 If an attempt to open a log file for the first time fails, the
997 disk log process terminates with the EXIT message {{failed,Rea‐
998 son},[{disk_log,open,1}]}. The function returns {error, Reason}
999 for all other errors.
1000 pid2name(Pid) -> {ok, Log} | undefined
1002 Types:
1004 Pid = pid()
1006 Log = log()
1007 Returns the log name given the pid of a disk log process on the
1009 current node, or undefined if the specified pid is not a disk
1010 log process.
1011 This function is meant to be used for debugging only.
1013 reopen(Log, File) -> ok | {error, reopen_error_rsn()}
1015 reopen(Log, File, Head) -> ok | {error, reopen_error_rsn()}
1017 breopen(Log, File, BHead) -> ok | {error, reopen_error_rsn()}
1019 Types:
1021 Log = log()
1023 File = file:filename()
1024 Head = term()
1025 BHead = iodata()
1026 reopen_error_rsn() =
1027 no_such_log |
1028 nonode |
1029 {read_only_mode, log()} |
1030 {blocked_log, log()} |
1031 {same_file_name, log()} |
1032 {invalid_index_file, file:filename()} |
1033 {invalid_header, invalid_header()} |
1034 {file_error, file:filename(), file_error()}
1035 Renames the log file to File and then recreates a new log file.
1037 If a wrap log exists, File is used as the base name of the
1038 renamed files. By default the header given to open/1 is written
1039 first in the newly opened log file, but if argument Head or
1040 BHead is specified, this item is used instead. The header argu‐
1041 ment is used only once. Next time a wrap log file is opened, the
1042 header given to open/1 is used.
1043 reopen/2,3 are used for internally formatted logs, and breopen/3
1045 for externally formatted logs.
1046 Owners subscribing to notifications receive a truncate message.
1048 Upon failure to reopen the log, the disk log process terminates
1050 with the EXIT message {{failed,Error},[{disk_log,Fun,Arity}]}.
1051 Other processes having requests queued receive the message
1052 {disk_log, Node, {error, disk_log_stopped}}.
1053 sync(Log) -> ok | {error, sync_error_rsn()}
1055 Types:
1057 Log = log()
1059 sync_error_rsn() =
1060 no_such_log |
1061 nonode |
1062 {read_only_mode, log()} |
1063 {blocked_log, log()} |
1064 {file_error, file:filename(), file_error()}
1065 Ensures that the contents of the log are written to the disk.
1067 This is usually a rather expensive operation.
1068 truncate(Log) -> ok | {error, trunc_error_rsn()}
1070 truncate(Log, Head) -> ok | {error, trunc_error_rsn()}
1072 btruncate(Log, BHead) -> ok | {error, trunc_error_rsn()}
1074 Types:
1076 Log = log()
1078 Head = term()
1079 BHead = iodata()
1080 trunc_error_rsn() =
1081 no_such_log |
1082 nonode |
1083 {read_only_mode, log()} |
1084 {blocked_log, log()} |
1085 {invalid_header, invalid_header()} |
1086 {file_error, file:filename(), file_error()}
1087 Removes all items from a disk log. If argument Head or BHead is
1089 specified, this item is written first in the newly truncated
1090 log, otherwise the header given to open/1 is used. The header
1091 argument is used only once. Next time a wrap log file is opened,
1092 the header given to open/1 is used.
1093 truncate/1,2 are used for internally formatted logs, and btrun‐
1095 cate/2 for externally formatted logs.
1096 Owners subscribing to notifications receive a truncate message.
1098 If the attempt to truncate the log fails, the disk log process
1100 terminates with the EXIT message {{failed,Rea‐
1101 son},[{disk_log,Fun,Arity}]}. Other processes having requests
1102 queued receive the message {disk_log, Node, {error,
1103 disk_log_stopped}}.
1104 unblock(Log) -> ok | {error, unblock_error_rsn()}
1106 Types:
1108 Log = log()
1110 unblock_error_rsn() =
1111 no_such_log |
1112 nonode |
1113 {not_blocked, log()} |
1114 {not_blocked_by_pid, log()}
1115 Unblocks a log. A log can only be unblocked by the blocking
1117 process.
1118
1120 file(3), pg2(3), wrap_log_reader(3)
1121Ericsson AB kernel 5.4.3.2 disk_log(3)