1disk_log(3)                Erlang Module Definition                disk_log(3)
2
3
4

NAME

6       disk_log - A disk-based term logging facility.
7

DESCRIPTION

9       disk_log  is a disk-based term logger that enables efficient logging of
10       items on files.
11
12       Two types of logs are supported:
13
14         halt logs:
15           Appends items to a single file, which size can be  limited  by  the
16           disk_log module.
17
18         wrap logs:
19           Uses  a  sequence  of wrap log files of limited size. As a wrap log
20           file is filled up, further items are logged on to the next file  in
21           the  sequence,  starting all over with the first file when the last
22           file is filled up.
23
24       For efficiency reasons, items are always written to files as binaries.
25
26       Two formats of the log files are supported:
27
28         internal format:
29           Supports automatic repair of log files that are not properly closed
30           and enables efficient reading of logged items in chunks using a set
31           of functions defined in this module. This is the only way  to  read
32           internally  formatted logs. An item logged to an internally format‐
33           ted log must not occupy more than 4 GB of disk space (the size must
34           fit in 4 bytes).
35
36         external format:
37           Leaves it up to the user to read and interpret the logged data. The
38           disk_log module cannot repair externally formatted logs.
39
40       For each open disk log, one process handles requests made to  the  disk
41       log.  This  process  is  created  when open/1 is called, provided there
42       exists no process handling the disk log. A process that  opens  a  disk
43       log can be an owner or an anonymous user of the disk log. Each owner is
44       linked to the disk log process, and an owner can  close  the  disk  log
45       either explicitly (by calling close/1 or lclose/1,2) or by terminating.
46
47       Owners  can subscribe to notifications, messages of the form {disk_log,
48       Node, Log, Info}, which are sent from the disk log process when certain
49       events  occur,  see  the  functions and in particular the open/1 option
50       notify. A log can have many owners, but a process cannot own a log more
51       than  once.  However,  the same process can open the log as a user more
52       than once.
53
54       For a disk log process to close its file  properly  and  terminate,  it
55       must  be  closed  by  its owners and once by some non-owner process for
56       each time the log was used anonymously. The users are counted and there
57       must not be any users left when the disk log process terminates.
58
59       Items  can  be  logged  synchronously by using functions log/2, blog/2,
60       log_terms/2, and blog_terms/2. For each of these functions, the  caller
61       is put on hold until the items are logged (but not necessarily written,
62       use sync/1 to ensure that). By adding an a to  each  of  the  mentioned
63       function  names,  we get functions that log items asynchronously. Asyn‐
64       chronous functions do not wait for the disk log process  to  write  the
65       items  to  the  file, but return the control to the caller more or less
66       immediately.
67
68       When  using  the  internal  format  for  logs,  use  functions   log/2,
69       log_terms/2,  alog/2, and alog_terms/2. These functions log one or more
70       Erlang terms. By  prefixing  each  of  the  functions  with  a  b  (for
71       "binary"),  we  get the corresponding blog() functions for the external
72       format. These functions log one or more chunks of bytes.  For  example,
73       to   log   the   string   "hello"   in   ASCII   format,  you  can  use
74       disk_log:blog(Log,        "hello"),        or        disk_log:blog(Log,
75       list_to_binary("hello")). The two alternatives are equally efficient.
76
77       The  blog()  functions  can also be used for internally formatted logs,
78       but in this case they must be called  with  binaries  constructed  with
79       calls  to  term_to_binary/1.  There  is  no check to ensure this, it is
80       entirely the responsibility of  the  caller.  If  these  functions  are
81       called  with  binaries  that  do  not  correspond  to Erlang terms, the
82       chunk/2,3 and automatic repair functions fail. The corresponding  terms
83       (not the binaries) are returned when chunk/2,3 is called.
84
85       A  collection of open disk logs with the same name running on different
86       nodes is said to be a distributed disk log if requests made to  any  of
87       the  logs are automatically made to the other logs as well. The members
88       of such a collection are called individual distributed  disk  logs,  or
89       just  distributed  disk logs if there is no risk of confusion. There is
90       no order between the members of such a collection. For example,  logged
91       terms  are  not  necessarily  written to the node where the request was
92       made before written to the other nodes. However, a few functions do not
93       make  requests  to all members of distributed disk logs, namely info/1,
94       chunk/2,3, bchunk/2,3, chunk_step/3, and lclose/1,2.
95
96       An open disk log that is not a distributed disk log is  said  to  be  a
97       local disk log. A local disk log is only accessible from the node where
98       the disk log process runs, whereas a distributed disk log is accessible
99       from  all  nodes  in  the Erlang system, except for those nodes where a
100       local disk log with the same name as the distributed disk  log  exists.
101       All  processes on nodes that have access to a local or distributed disk
102       log can log items or otherwise change, inspect, or close the log.
103
104       It is not guaranteed that all log files of a distributed disk log  con‐
105       tain the same log items. No attempt is made to synchronize the contents
106       of the files. However, as long as at least one of the involved nodes is
107       alive  at each time, all items are logged. When logging items to a dis‐
108       tributed log, or otherwise trying to change the log, the  replies  from
109       individual  logs are ignored. If all nodes are down, the disk log func‐
110       tions reply with a nonode error.
111
112   Note:
113       In some applications, it can be unacceptable that replies from individ‐
114       ual  logs are ignored. An alternative in such situations is to use many
115       local disk logs instead of one distributed disk log, and implement  the
116       distribution without use of the disk_log module.
117
118
119       Errors are reported differently for asynchronous log attempts and other
120       uses of the disk_log  module.  When  used  synchronously,  this  module
121       replies  with  an  error  message, but when called asynchronously, this
122       module does not know where to send the error message.  Instead,  owners
123       subscribing to notifications receive an error_status message.
124
125       The  disk_log module does not report errors to the error_logger module.
126       It is up to the caller to decide whether to employ  the  error  logger.
127       Function  format_error/1  can be used to produce readable messages from
128       error replies. However, information events are sent to the error logger
129       in  two  situations,  namely  when a log is repaired, or when a file is
130       missing while reading chunks.
131
132       Error message no_such_log means that the  specified  disk  log  is  not
133       open. Nothing is said about whether the disk log files exist or not.
134
135   Note:
136       If  an  attempt  to  reopen or truncate a log fails (see reopen/2,3 and
137       truncate/1,2) the disk log process terminates immediately.  Before  the
138       process  terminates,  links  to  owners  and  blocking  processes  (see
139       block/1,2) are removed. The effect is that the links work in one direc‐
140       tion  only.  Any  process using a disk log must check for error message
141       no_such_log if some other process truncates or reopens the log simulta‐
142       neously.
143
144

DATA TYPES

146       log() = term()
147
148       dlog_size() =
149           infinity |
150           integer() >= 1 |
151           {MaxNoBytes :: integer() >= 1, MaxNoFiles :: integer() >= 1}
152
153       dlog_format() = external | internal
154
155       dlog_head_opt() = none | term() | iodata()
156
157       dlog_mode() = read_only | read_write
158
159       dlog_type() = halt | wrap
160
161       continuation()
162
163              Chunk   continuation   returned  by  chunk/2,3,  bchunk/2,3,  or
164              chunk_step/3.
165
166       invalid_header() = term()
167
168       file_error() = term()
169

EXPORTS

171       accessible_logs() -> {[LocalLog], [DistributedLog]}
172
173              Types:
174
175                 LocalLog = DistributedLog = log()
176
177              Returns the names of the disk logs  accessible  on  the  current
178              node.  The  first  list  contains local disk logs and the second
179              list contains distributed disk logs.
180
181       alog(Log, Term) -> notify_ret()
182
183       balog(Log, Bytes) -> notify_ret()
184
185              Types:
186
187                 Log = log()
188                 Term = term()
189                 Bytes = iodata()
190                 notify_ret() = ok | {error, no_such_log}
191
192              Asynchronously append an item to a disk log. alog/2 is used  for
193              internally  formatted  logs and balog/2 for externally formatted
194              logs. balog/2 can also be used for internally formatted logs  if
195              the binary is constructed with a call to term_to_binary/1.
196
197              Owners  subscribing  to notifications receive message read_only,
198              blocked_log, or format_external if the item cannot be written on
199              the  log,  and  possibly  one  of  the  messages  wrap, full, or
200              error_status  if  an  item  is  written  on  the  log.   Message
201              error_status is sent if something is wrong with the header func‐
202              tion or if a file error occurs.
203
204       alog_terms(Log, TermList) -> notify_ret()
205
206       balog_terms(Log, ByteList) -> notify_ret()
207
208              Types:
209
210                 Log = log()
211                 TermList = [term()]
212                 ByteList = [iodata()]
213                 notify_ret() = ok | {error, no_such_log}
214
215              Asynchronously  append  a  list  of  items  to   a   disk   log.
216              alog_terms/2   is   used   for  internally  formatted  logs  and
217              balog_terms/2 for externally formatted logs.  balog_terms/2  can
218              also  be  used for internally formatted logs if the binaries are
219              constructed with calls to term_to_binary/1.
220
221              Owners subscribing to notifications receive  message  read_only,
222              blocked_log,  or  format_external if the items cannot be written
223              on the log, and possibly one or more of the messages wrap, full,
224              and  error_status  if  items  are  written  on  the log. Message
225              error_status is sent if something is wrong with the header func‐
226              tion or if a file error occurs.
227
228       block(Log) -> ok | {error, block_error_rsn()}
229
230       block(Log, QueueLogRecords) -> ok | {error, block_error_rsn()}
231
232              Types:
233
234                 Log = log()
235                 QueueLogRecords = boolean()
236                 block_error_rsn() = no_such_log | nonode | {blocked_log, log()}
237
238              With  a  call  to  block/1,2  a  process can block a log. If the
239              blocking process is not an owner of the log, a temporary link is
240              created  between  the disk log process and the blocking process.
241              The link ensures that the disk log is unblocked if the  blocking
242              process terminates without first closing or unblocking the log.
243
244              Any process can probe a blocked log with info/1 or close it with
245              close/1. The blocking process can also use functions  chunk/2,3,
246              bchunk/2,3,  chunk_step/3,  and unblock/1 without being affected
247              by the block. Any other attempt than those mentioned so  far  to
248              update  or read a blocked log suspends the calling process until
249              the log is unblocked  or  returns  error  message  {blocked_log,
250              Log},  depending on whether the value of QueueLogRecords is true
251              or false. QueueLogRecords defaults to true,  which  is  used  by
252              block/1.
253
254       change_header(Log, Header) -> ok | {error, Reason}
255
256              Types:
257
258                 Log = log()
259                 Header =
260                     {head, dlog_head_opt()} |
261                     {head_func, MFA :: {atom(), atom(), list()}}
262                 Reason =
263                     no_such_log | nonode |
264                     {read_only_mode, Log} |
265                     {blocked_log, Log} |
266                     {badarg, head}
267
268              Changes  the value of option head or head_func for an owner of a
269              disk log.
270
271       change_notify(Log, Owner, Notify) -> ok | {error, Reason}
272
273              Types:
274
275                 Log = log()
276                 Owner = pid()
277                 Notify = boolean()
278                 Reason =
279                     no_such_log | nonode |
280                     {blocked_log, Log} |
281                     {badarg, notify} |
282                     {not_owner, Owner}
283
284              Changes the value of option notify for an owner of a disk log.
285
286       change_size(Log, Size) -> ok | {error, Reason}
287
288              Types:
289
290                 Log = log()
291                 Size = dlog_size()
292                 Reason =
293                     no_such_log | nonode |
294                     {read_only_mode, Log} |
295                     {blocked_log, Log} |
296                     {new_size_too_small, Log, CurrentSize :: integer() >=  1}
297                 |
298                     {badarg, size} |
299                     {file_error, file:filename(), file_error()}
300
301              Changes  the  size  of an open log. For a halt log, the size can
302              always be increased, but it cannot  be  decreased  to  something
303              less than the current file size.
304
305              For a wrap log, both the size and the number of files can always
306              be increased, as long as the number of  files  does  not  exceed
307              65000.  If  the maximum number of files is decreased, the change
308              is not valid until the current file is full and the log wraps to
309              the next file. The redundant files are removed the next time the
310              log wraps around, that is, starts to log to file number 1.
311
312              As an example, assume that the old maximum number of files is 10
313              and  that  the  new maximum number of files is 6. If the current
314              file number is not greater than the new maximum number of files,
315              files 7-10 are removed when file 6 is full and the log starts to
316              write to file number 1 again. Otherwise, the files greater  than
317              the  current file are removed when the current file is full (for
318              example, if the current file is 8, files 9 and 10 are  removed).
319              The  files  between the new maximum number of files and the cur‐
320              rent file (that is, files 7 and 8) are  removed  the  next  time
321              file 6 is full.
322
323              If  the  size  of the files is decreased, the change immediately
324              affects the current log. It does not  change  the  size  of  log
325              files already full until the next time they are used.
326
327              If  the log size is decreased, for example, to save space, func‐
328              tion inc_wrap_file/1 can be used to force the log to wrap.
329
330       chunk(Log, Continuation) -> chunk_ret()
331
332       chunk(Log, Continuation, N) -> chunk_ret()
333
334       bchunk(Log, Continuation) -> bchunk_ret()
335
336       bchunk(Log, Continuation, N) -> bchunk_ret()
337
338              Types:
339
340                 Log = log()
341                 Continuation = start | continuation()
342                 N = integer() >= 1 | infinity
343                 chunk_ret() =
344                     {Continuation2 :: continuation(), Terms :: [term()]} |
345                     {Continuation2 :: continuation(),
346                      Terms :: [term()],
347                      Badbytes :: integer() >= 0} |
348                     eof |
349                     {error, Reason :: chunk_error_rsn()}
350                 bchunk_ret() =
351                     {Continuation2 :: continuation(), Binaries :: [binary()]} |
352                     {Continuation2 :: continuation(),
353                      Binaries :: [binary()],
354                      Badbytes :: integer() >= 0} |
355                     eof |
356                     {error, Reason :: chunk_error_rsn()}
357                 chunk_error_rsn() =
358                     no_such_log |
359                     {format_external, log()} |
360                     {blocked_log, log()} |
361                     {badarg, continuation} |
362                     {not_internal_wrap, log()} |
363                     {corrupt_log_file, FileName :: file:filename()} |
364                     {file_error, file:filename(), file_error()}
365
366              Efficiently reads the terms that are appended to  an  internally
367              formatted  log.  It  minimizes  disk  I/O by reading 64 kilobyte
368              chunks from the file. Functions bchunk/2,3 return  the  binaries
369              read  from  the  file,  they do not call binary_to_term(). Apart
370              from that, they work just like chunk/2,3.
371
372              The first time chunk() (or bchunk()) is called, an initial  con‐
373              tinuation,  the  atom  start,  must  be  provided. If a disk log
374              process is running on the current node, terms are read from that
375              log. Otherwise, an individual distributed log on some other node
376              is chosen, if such a log exists.
377
378              When chunk/3 is called, N controls the maximum number  of  terms
379              that  are read from the log in each chunk. Defaults to infinity,
380              which means that all the terms  contained  in  the  64  kilobyte
381              chunk are read. If less than N terms are returned, this does not
382              necessarily mean that the end of the file is reached.
383
384              chunk() returns a tuple {Continuation2, Terms}, where Terms is a
385              list  of  terms  found  in the log. Continuation2 is yet another
386              continuation, which must be passed on to any subsequent calls to
387              chunk(). With a series of calls to chunk(), all terms from a log
388              can be extracted.
389
390              chunk() returns a tuple {Continuation2, Terms, Badbytes} if  the
391              log  is  opened in read-only mode and the read chunk is corrupt.
392              Badbytes is the number of bytes in the  file  found  not  to  be
393              Erlang  terms in the chunk. Notice that the log is not repaired.
394              When trying to read chunks from a log opened in read-write mode,
395              tuple {corrupt_log_file, FileName} is returned if the read chunk
396              is corrupt.
397
398              chunk() returns eof when the end  of  the  log  is  reached,  or
399              {error,  Reason} if an error occurs. If a wrap log file is miss‐
400              ing, a message is output on the error log.
401
402              When chunk/2,3 is used with wrap logs, the returned continuation
403              might  not be valid in the next call to chunk(). This is because
404              the log can wrap and delete the file into which the continuation
405              points.  To  prevent  this,  the  log  can be blocked during the
406              search.
407
408       chunk_info(Continuation) -> InfoList | {error, Reason}
409
410              Types:
411
412                 Continuation = continuation()
413                 InfoList = [{node, Node :: node()}, ...]
414                 Reason = {no_continuation, Continuation}
415
416              Returns the pair {node, Node}, describing the chunk continuation
417              returned by chunk/2,3, bchunk/2,3, or chunk_step/3.
418
419              Terms are read from the disk log running on Node.
420
421       chunk_step(Log, Continuation, Step) ->
422                     {ok, any()} | {error, Reason}
423
424              Types:
425
426                 Log = log()
427                 Continuation = start | continuation()
428                 Step = integer()
429                 Reason =
430                     no_such_log | end_of_log |
431                     {format_external, Log} |
432                     {blocked_log, Log} |
433                     {badarg, continuation} |
434                     {file_error, file:filename(), file_error()}
435
436              Can  be  used with chunk/2,3 and bchunk/2,3 to search through an
437              internally formatted wrap log. It takes as argument a  continua‐
438              tion  as returned by chunk/2,3, bchunk/2,3, or chunk_step/3, and
439              steps forward (or backward) Step files in the wrap log. The con‐
440              tinuation returned, points to the first log item in the new cur‐
441              rent file.
442
443              If atom start is specified as continuation, a disk log  to  read
444              terms  from  is  chosen.  A local or distributed disk log on the
445              current node is preferred to an individual  distributed  log  on
446              some other node.
447
448              If  the wrap log is not full because all files are not yet used,
449              {error, end_of_log} is returned if trying to  step  outside  the
450              log.
451
452       close(Log) -> ok | {error, close_error_rsn()}
453
454              Types:
455
456                 Log = log()
457                 close_error_rsn() =
458                     no_such_log | nonode |
459                     {file_error, file:filename(), file_error()}
460
461              Closes  a  local or distributed disk log properly. An internally
462              formatted log  must  be  closed  before  the  Erlang  system  is
463              stopped.  Otherwise,  the  log  is  regarded as unclosed and the
464              automatic repair procedure is activated next  time  the  log  is
465              opened.
466
467              The disk log process is not terminated as long as there are own‐
468              ers or users of the log. All owners must close the log, possibly
469              by  terminating. Also, any other process, not only the processes
470              that have opened the log anonymously, can  decrement  the  users
471              counter by closing the log. Attempts to close a log by a process
472              that is not an owner are ignored if there are no users.
473
474              If the log is blocked by the closing process, the  log  is  also
475              unblocked.
476
477       format_error(Error) -> io_lib:chars()
478
479              Types:
480
481                 Error = term()
482
483              Given  the  error  returned by any function in this module, this
484              function returns a descriptive string of the error  in  English.
485              For  file  errors,  function  format_error/1  in  module file is
486              called.
487
488       inc_wrap_file(Log) -> ok | {error, inc_wrap_error_rsn()}
489
490              Types:
491
492                 Log = log()
493                 inc_wrap_error_rsn() =
494                     no_such_log | nonode |
495                     {read_only_mode, log()} |
496                     {blocked_log, log()} |
497                     {halt_log, log()} |
498                     {invalid_header, invalid_header()} |
499                     {file_error, file:filename(), file_error()}
500                 invalid_header() = term()
501
502              Forces the internally formatted disk log to start logging to the
503              next  log  file. It can be used, for example, with change_size/2
504              to reduce the amount of disk space allocated by the disk log.
505
506              Owners subscribing to notifications normally receive a wrap mes‐
507              sage, but if an error occurs with a reason tag of invalid_header
508              or file_error, an error_status message is sent.
509
510       info(Log) -> InfoList | {error, no_such_log}
511
512              Types:
513
514                 Log = log()
515                 InfoList = [dlog_info()]
516                 dlog_info() =
517                     {name, Log :: log()} |
518                     {file, File :: file:filename()} |
519                     {type, Type :: dlog_type()} |
520                     {format, Format :: dlog_format()} |
521                     {size, Size :: dlog_size()} |
522                     {mode, Mode :: dlog_mode()} |
523                     {owners, [{pid(), Notify :: boolean()}]} |
524                     {users, Users :: integer() >= 0} |
525                     {status,
526                      Status :: ok | {blocked, QueueLogRecords :: boolean()}} |
527                     {node, Node :: node()} |
528                     {distributed, Dist :: local | [node()]} |
529                     {head,
530                      Head ::
531                          none | {head, term()} | (MFA :: {atom(), atom(), list()})} |
532                     {no_written_items, NoWrittenItems :: integer() >= 0} |
533                     {full, Full :: boolean} |
534                     {no_current_bytes, integer() >= 0} |
535                     {no_current_items, integer() >= 0} |
536                     {no_items, integer() >= 0} |
537                     {current_file, integer() >= 1} |
538                     {no_overflows,
539                      {SinceLogWasOpened :: integer() >= 0,
540                       SinceLastInfo :: integer() >= 0}}
541
542              Returns a list of {Tag, Value} pairs describing the  log.  If  a
543              disk  log  process  is  running on the current node, that log is
544              used as source of information, otherwise an individual  distrib‐
545              uted log on some other node is chosen, if such a log exists.
546
547              The following pairs are returned for all logs:
548
549                {name, Log}:
550                  Log is the log name as specified by the open/1 option name.
551
552                {file, File}:
553                  For  halt  logs File is the filename, and for wrap logs File
554                  is the base name.
555
556                {type, Type}:
557                  Type is the log type as specified by the open/1 option type.
558
559                {format, Format}:
560                  Format is the log format as specified by the  open/1  option
561                  format.
562
563                {size, Size}:
564                  Size is the log size as specified by the open/1 option size,
565                  or  the  size  set  by  change_size/2.  The  value  set   by
566                  change_size/2 is reflected immediately.
567
568                {mode, Mode}:
569                  Mode is the log mode as specified by the open/1 option mode.
570
571                {owners, [{pid(), Notify}]}:
572                  Notify is the value set by the open/1 option notify or func‐
573                  tion change_notify/3 for the owners of the log.
574
575                {users, Users}:
576                  Users is the number of anonymous users of the log,  see  the
577                  open/1 option linkto.
578
579                {status, Status}:
580                  Status  is  ok or {blocked, QueueLogRecords} as set by func‐
581                  tions block/1,2 and unblock/1.
582
583                {node, Node}:
584                  The information returned by the current invocation of  func‐
585                  tion info/1 is gathered from the disk log process running on
586                  Node.
587
588                {distributed, Dist}:
589                  If the log is local on the current node, Dist has the  value
590                  local,  otherwise all nodes where the log is distributed are
591                  returned as a list.
592
593              The  following  pairs  are  returned  for  all  logs  opened  in
594              read_write mode:
595
596                {head, Head}:
597                  Depending  on  the  value  of  the  open/1  options head and
598                  head_func, or set by function change_header/2, the value  of
599                  Head  is none (default), {head, H} (head option), or {M,F,A}
600                  (head_func option).
601
602                {no_written_items, NoWrittenItems}:
603                  NoWrittenItems is the number of items  written  to  the  log
604                  since the disk log process was created.
605
606              The   following  pair  is  returned  for  halt  logs  opened  in
607              read_write mode:
608
609                {full, Full}:
610                  Full is true or false depending on whether the halt  log  is
611                  full or not.
612
613              The  following  pairs  are  returned  for  wrap  logs  opened in
614              read_write mode:
615
616                {no_current_bytes, integer() >= 0}:
617                  The number of bytes written to the current wrap log file.
618
619                {no_current_items, integer() >= 0}:
620                  The number of items written to the current  wrap  log  file,
621                  header inclusive.
622
623                {no_items, integer() >= 0}:
624                  The total number of items in all wrap log files.
625
626                {current_file, integer()}:
627                  The  ordinal  for  the  current  wrap  log file in the range
628                  1..MaxNoFiles, where MaxNoFiles is specified by  the  open/1
629                  option size or set by change_size/2.
630
631                {no_overflows, {SinceLogWasOpened, SinceLastInfo}}:
632                  SinceLogWasOpened  (SinceLastInfo)  is the number of times a
633                  wrap log file has been filled up and a new one is opened  or
634                  inc_wrap_file/1  has been called since the disk log was last
635                  opened (info/1 was last called). The first  time  info/2  is
636                  called after a log was (re)opened or truncated, the two val‐
637                  ues are equal.
638
639              Notice that functions chunk/2,3, bchunk/2,3, and chunk_step/3 do
640              not affect any value returned by info/1.
641
642       lclose(Log) -> ok | {error, lclose_error_rsn()}
643
644       lclose(Log, Node) -> ok | {error, lclose_error_rsn()}
645
646              Types:
647
648                 Log = log()
649                 Node = node()
650                 lclose_error_rsn() =
651                     no_such_log | {file_error, file:filename(), file_error()}
652
653              lclose/1  closes a local log or an individual distributed log on
654              the current node.
655
656              lclose/2 closes an individual distributed log on  the  specified
657              node if the node is not the current one.
658
659              lclose(Log)  is  equivalent  to  lclose(Log,  node()).  See also
660              close/1.
661
662              If no log with the specified name exist on the  specified  node,
663              no_such_log is returned.
664
665       log(Log, Term) -> ok | {error, Reason :: log_error_rsn()}
666
667       blog(Log, Bytes) -> ok | {error, Reason :: log_error_rsn()}
668
669              Types:
670
671                 Log = log()
672                 Term = term()
673                 Bytes = iodata()
674                 log_error_rsn() =
675                     no_such_log | nonode |
676                     {read_only_mode, log()} |
677                     {format_external, log()} |
678                     {blocked_log, log()} |
679                     {full, log()} |
680                     {invalid_header, invalid_header()} |
681                     {file_error, file:filename(), file_error()}
682
683              Synchronously  appends  a  term  to  a  disk  log. Returns ok or
684              {error, Reason} when the term is written to disk. If the log  is
685              distributed,  ok  is  returned, unless all nodes are down. Terms
686              are written by the ordinary write() function  of  the  operating
687              system.  Hence, it is not guaranteed that the term is written to
688              disk, it can linger in the operating system kernel for a  while.
689              To ensure that the item is written to disk, function sync/1 must
690              be called.
691
692              log/2 is used for internally  formatted  logs,  and  blog/2  for
693              externally  formatted  logs.  blog/2 can also be used for inter‐
694              nally formatted logs if the binary is constructed with a call to
695              term_to_binary/1.
696
697              Owners  subscribing  to  notifications  are notified of an error
698              with  an  error_status  message  if  the  error  reason  tag  is
699              invalid_header or file_error.
700
701       log_terms(Log, TermList) ->
702                    ok | {error, Resaon :: log_error_rsn()}
703
704       blog_terms(Log, BytesList) ->
705                     ok | {error, Reason :: log_error_rsn()}
706
707              Types:
708
709                 Log = log()
710                 TermList = [term()]
711                 BytesList = [iodata()]
712                 log_error_rsn() =
713                     no_such_log | nonode |
714                     {read_only_mode, log()} |
715                     {format_external, log()} |
716                     {blocked_log, log()} |
717                     {full, log()} |
718                     {invalid_header, invalid_header()} |
719                     {file_error, file:filename(), file_error()}
720
721              Synchronously  appends  a  list  of items to the log. It is more
722              efficient to use these functions instead of functions log/2  and
723              blog/2.  The  specified  list is split into as large sublists as
724              possible (limited by the size of wrap log files), and each  sub‐
725              list is logged as one single item, which reduces the overhead.
726
727              log_terms/2   is   used   for  internally  formatted  logs,  and
728              blog_terms/2 for externally  formatted  logs.  blog_terms/2  can
729              also  be  used for internally formatted logs if the binaries are
730              constructed with calls to term_to_binary/1.
731
732              Owners subscribing to notifications are  notified  of  an  error
733              with  an  error_status  message  if  the  error  reason  tag  is
734              invalid_header or file_error.
735
736       open(ArgL) -> open_ret() | dist_open_ret()
737
738              Types:
739
740                 ArgL = dlog_options()
741                 dlog_options() = [dlog_option()]
742                 dlog_option() =
743                     {name, Log :: log()} |
744                     {file, FileName :: file:filename()} |
745                     {linkto, LinkTo :: none | pid()} |
746                     {repair, Repair :: true | false | truncate} |
747                     {type, Type :: dlog_type()} |
748                     {format, Format :: dlog_format()} |
749                     {size, Size :: dlog_size()} |
750                     {distributed, Nodes :: [node()]} |
751                     {notify, boolean()} |
752                     {head, Head :: dlog_head_opt()} |
753                     {head_func, MFA :: {atom(), atom(), list()}} |
754                     {quiet, boolean()} |
755                     {mode, Mode :: dlog_mode()}
756                 open_ret() = ret() | {error, open_error_rsn()}
757                 ret() =
758                     {ok, Log :: log()} |
759                     {repaired,
760                      Log :: log(),
761                      {recovered, Rec :: integer() >= 0},
762                      {badbytes, Bad :: integer() >= 0}}
763                 dist_open_ret() =
764                     {[{node(), ret()}], [{node(), {error, dist_error_rsn()}}]}
765                 dist_error_rsn() = nodedown | open_error_rsn()
766                 open_error_rsn() =
767                     no_such_log |
768                     {badarg, term()} |
769                     {size_mismatch,
770                      CurrentSize :: dlog_size(),
771                      NewSize :: dlog_size()} |
772                     {arg_mismatch,
773                      OptionName :: dlog_optattr(),
774                      CurrentValue :: term(),
775                      Value :: term()} |
776                     {name_already_open, Log :: log()} |
777                     {open_read_write, Log :: log()} |
778                     {open_read_only, Log :: log()} |
779                     {need_repair, Log :: log()} |
780                     {not_a_log_file, FileName :: file:filename()} |
781                     {invalid_index_file, FileName :: file:filename()} |
782                     {invalid_header, invalid_header()} |
783                     {file_error, file:filename(), file_error()} |
784                     {node_already_open, Log :: log()}
785                 dlog_optattr() =
786                     name | file | linkto | repair | type | format | size |
787                     distributed | notify | head | head_func | mode
788                 dlog_size() =
789                     infinity |
790                     integer() >= 1 |
791                     {MaxNoBytes :: integer() >= 1, MaxNoFiles :: integer() >= 1}
792
793              Parameter ArgL is a list of the following options:
794
795                {name, Log}:
796                  Specifies the log name. This name must be  passed  on  as  a
797                  parameter  in all subsequent logging operations. A name must
798                  always be supplied.
799
800                {file, FileName}:
801                  Specifies the name of the file to be used for logged  terms.
802                  If  this  value  is omitted and the log name is an atom or a
803                  string, the filename defaults to lists:concat([Log, ".LOG"])
804                  for halt logs.
805
806                  For wrap logs, this is the base name of the files. Each file
807                  in a wrap log is called <base_name>.N, where N is  an  inte‐
808                  ger. Each wrap log also has two files called <base_name>.idx
809                  and <base_name>.siz.
810
811                {linkto, LinkTo}:
812                  If LinkTo is a pid, it becomes  an  owner  of  the  log.  If
813                  LinkTo  is none, the log records that it is used anonymously
814                  by some  process  by  incrementing  the  users  counter.  By
815                  default, the process that calls open/1 owns the log.
816
817                {repair, Repair}:
818                  If  Repair  is  true,  the  current log file is repaired, if
819                  needed. As the restoration is initiated, a message is output
820                  on the error log. If false is specified, no automatic repair
821                  is attempted. Instead, the tuple {error, {need_repair, Log}}
822                  is  returned  if  an  attempt  is made to open a corrupt log
823                  file. If truncate is specified, the log file  becomes  trun‐
824                  cated, creating an empty log. Defaults to true, which has no
825                  effect on logs opened in read-only mode.
826
827                {type, Type}:
828                  The log type. Defaults to halt.
829
830                {format, Format}:
831                  Disk log format. Defaults to internal.
832
833                {size, Size}:
834                  Log size.
835
836                  When a halt log has reached its maximum size,  all  attempts
837                  to  log more items are rejected. Defaults to infinity, which
838                  for halt implies that there is no maximum size.
839
840                  For wrap logs, parameter Size can  be  a  pair  {MaxNoBytes,
841                  MaxNoFiles} or infinity. In the latter case, if the files of
842                  an existing wrap log with the same name can  be  found,  the
843                  size  is read from the existing wrap log, otherwise an error
844                  is returned.
845
846                  Wrap logs write at most MaxNoBytes bytes on  each  file  and
847                  use MaxNoFiles files before starting all over with the first
848                  wrap log file. Regardless of MaxNoBytes, at least the header
849                  (if  there is one) and one item are written on each wrap log
850                  file before wrapping to the next file.
851
852                  When opening an existing wrap log, it is  not  necessary  to
853                  supply  a value for option Size, but any supplied value must
854                  equal the current log  size,  otherwise  the  tuple  {error,
855                  {size_mismatch, CurrentSize, NewSize}} is returned.
856
857                {distributed, Nodes}:
858                  This  option can be used for adding members to a distributed
859                  disk log. Defaults to [], which means that the log is  local
860                  on the current node.
861
862                {notify, boolean()}:
863                  If true, the log owners are notified when certain log events
864                  occur. Defaults to false. The owners are  sent  one  of  the
865                  following messages when an event occurs:
866
867                  {disk_log, Node, Log, {wrap, NoLostItems}}:
868                    Sent  when a wrap log has filled up one of its files and a
869                    new file is opened. NoLostItems is the  number  of  previ‐
870                    ously logged items that were lost when truncating existing
871                    files.
872
873                  {disk_log, Node, Log, {truncated, NoLostItems}}:
874                    Sent when a log is truncated or reopened.  For  halt  logs
875                    NoLostItems  is  the  number  of  items written on the log
876                    since the disk log process  was  created.  For  wrap  logs
877                    NoLostItems is the number of items on all wrap log files.
878
879                  {disk_log, Node, Log, {read_only, Items}}:
880                    Sent  when  an  asynchronous  log attempt is made to a log
881                    file opened in read-only mode. Items is the items from the
882                    log attempt.
883
884                  {disk_log, Node, Log, {blocked_log, Items}}:
885                    Sent when an asynchronous log attempt is made to a blocked
886                    log that does not queue log attempts. Items is  the  items
887                    from the log attempt.
888
889                  {disk_log, Node, Log, {format_external, Items}}:
890                    Sent  when  function  alog/2  or  alog_terms/2 is used for
891                    internally formatted logs. Items is the items from the log
892                    attempt.
893
894                  {disk_log, Node, Log, full}:
895                    Sent  when  an  attempt  to  log items to a wrap log would
896                    write more bytes than the limit set by option size.
897
898                  {disk_log, Node, Log, {error_status, Status}}:
899                    Sent when the error status changes. The  error  status  is
900                    defined by the outcome of the last attempt to log items to
901                    the log, or to truncate the log, or the last use of  func‐
902                    tion  sync/1, inc_wrap_file/1, or change_size/2. Status is
903                    either ok or {error, Error}, the  former  is  the  initial
904                    value.
905
906                {head, Head}:
907                  Specifies  a  header to be written first on the log file. If
908                  the log is a wrap log, the item Head  is  written  first  in
909                  each  new file. Head is to be a term if the format is inter‐
910                  nal, otherwise a sequence of bytes. Defaults to none,  which
911                  means that no header is written first on the file.
912
913                {head_func, {M,F,A}}:
914                  Specifies  a  function to be called each time a new log file
915                  is opened. The call M:F(A) is assumed to return {ok,  Head}.
916                  The item Head is written first in each file. Head is to be a
917                  term if the format is  internal,  otherwise  a  sequence  of
918                  bytes.
919
920                {mode, Mode}:
921                  Specifies  if  the log is to be opened in read-only or read-
922                  write mode. Defaults to read_write.
923
924                {quiet, Boolean}:
925                  Specifies if messages will be sent to error_logger on recov‐
926                  erable errors with the log files. Defaults to false.
927
928              open/1 returns {ok, Log} if the log file is successfully opened.
929              If the file is successfully repaired, the tuple {repaired,  Log,
930              {recovered, Rec}, {badbytes, Bad}} is returned, where Rec is the
931              number of whole Erlang terms found in the file and  Bad  is  the
932              number  of  bytes  in the file that are non-Erlang terms. If the
933              parameter distributed is specified, open/1  returns  a  list  of
934              successful  replies  and a list of erroneous replies. Each reply
935              is tagged with the node name.
936
937              When a disk log is opened in read-write mode, any  existing  log
938              file  is  checked for. If there is none, a new empty log is cre‐
939              ated, otherwise the existing file  is  opened  at  the  position
940              after the last logged item, and the logging of items starts from
941              there. If the format is internal and the existing  file  is  not
942              recognized  as  an  internally  formatted  log,  a tuple {error,
943              {not_a_log_file, FileName}} is returned.
944
945              open/1 cannot be used for changing the values of options  of  an
946              open  log.  When  there  are prior owners or users of a log, all
947              option values except name, linkto, and notify are  only  checked
948              against  the values supplied before as option values to function
949              open/1,  change_header/2,  change_notify/3,  or   change_size/2.
950              Thus,  none  of  the  options  except name is mandatory. If some
951              specified value differs from the current value, a tuple  {error,
952              {arg_mismatch, OptionName, CurrentValue, Value}} is returned.
953
954          Note:
955              If  an  owner  attempts to open a log as owner once again, it is
956              acknowledged with the return value {ok, Log}, but the  state  of
957              the disk log is not affected.
958
959
960              If  a  log  with a specified name is local on some node, and one
961              tries to open the log distributed on the same  node,  the  tuple
962              {error, {node_already_open, Log}} is returned. The same tuple is
963              returned if the log is distributed on some node, and  one  tries
964              to  open  the  log  locally on the same node. Opening individual
965              distributed disk logs for the first time adds those  logs  to  a
966              (possibly  empty) distributed disk log. The supplied option val‐
967              ues are used on all nodes mentioned by option distributed. Indi‐
968              vidual  distributed  logs know nothing about each other's option
969              values, so each node can be given unique option values by creat‐
970              ing a distributed log with many calls to open/1.
971
972              A log file can be opened more than once by giving different val‐
973              ues to option name or by using the same file when distributing a
974              log  on different nodes. It is up to the user of module disk_log
975              to ensure that not more than one  disk  log  process  has  write
976              access to any file, otherwise the file can be corrupted.
977
978              If  an  attempt to open a log file for the first time fails, the
979              disk log process terminates with the EXIT message  {{failed,Rea‐
980              son},[{disk_log,open,1}]}.  The function returns {error, Reason}
981              for all other errors.
982
983       pid2name(Pid) -> {ok, Log} | undefined
984
985              Types:
986
987                 Pid = pid()
988                 Log = log()
989
990              Returns the log name given the pid of a disk log process on  the
991              current  node,  or  undefined if the specified pid is not a disk
992              log process.
993
994              This function is meant to be used for debugging only.
995
996       reopen(Log, File) -> ok | {error, reopen_error_rsn()}
997
998       reopen(Log, File, Head) -> ok | {error, reopen_error_rsn()}
999
1000       breopen(Log, File, BHead) -> ok | {error, reopen_error_rsn()}
1001
1002              Types:
1003
1004                 Log = log()
1005                 File = file:filename()
1006                 Head = term()
1007                 BHead = iodata()
1008                 reopen_error_rsn() =
1009                     no_such_log | nonode |
1010                     {read_only_mode, log()} |
1011                     {blocked_log, log()} |
1012                     {same_file_name, log()} |
1013                     {invalid_index_file, file:filename()} |
1014                     {invalid_header, invalid_header()} |
1015                     {file_error, file:filename(), file_error()}
1016
1017              Renames the log file to File and then recreates a new log  file.
1018              If  a  wrap  log  exists,  File  is used as the base name of the
1019              renamed files. By default the header given to open/1 is  written
1020              first  in  the  newly  opened  log file, but if argument Head or
1021              BHead is specified, this item is used instead. The header  argu‐
1022              ment is used only once. Next time a wrap log file is opened, the
1023              header given to open/1 is used.
1024
1025              reopen/2,3 are used for internally formatted logs, and breopen/3
1026              for externally formatted logs.
1027
1028              Owners subscribing to notifications receive a truncate message.
1029
1030              Upon  failure to reopen the log, the disk log process terminates
1031              with the EXIT  message  {{failed,Error},[{disk_log,Fun,Arity}]}.
1032              Other  processes  having  requests  queued  receive  the message
1033              {disk_log, Node, {error, disk_log_stopped}}.
1034
1035       sync(Log) -> ok | {error, sync_error_rsn()}
1036
1037              Types:
1038
1039                 Log = log()
1040                 sync_error_rsn() =
1041                     no_such_log | nonode |
1042                     {read_only_mode, log()} |
1043                     {blocked_log, log()} |
1044                     {file_error, file:filename(), file_error()}
1045
1046              Ensures that the contents of the log are written  to  the  disk.
1047              This is usually a rather expensive operation.
1048
1049       truncate(Log) -> ok | {error, trunc_error_rsn()}
1050
1051       truncate(Log, Head) -> ok | {error, trunc_error_rsn()}
1052
1053       btruncate(Log, BHead) -> ok | {error, trunc_error_rsn()}
1054
1055              Types:
1056
1057                 Log = log()
1058                 Head = term()
1059                 BHead = iodata()
1060                 trunc_error_rsn() =
1061                     no_such_log | nonode |
1062                     {read_only_mode, log()} |
1063                     {blocked_log, log()} |
1064                     {invalid_header, invalid_header()} |
1065                     {file_error, file:filename(), file_error()}
1066
1067              Removes  all items from a disk log. If argument Head or BHead is
1068              specified, this item is written first  in  the  newly  truncated
1069              log,  otherwise  the  header given to open/1 is used. The header
1070              argument is used only once. Next time a wrap log file is opened,
1071              the header given to open/1 is used.
1072
1073              truncate/1,2  are used for internally formatted logs, and btrun‐
1074              cate/2 for externally formatted logs.
1075
1076              Owners subscribing to notifications receive a truncate message.
1077
1078              If the attempt to truncate the log fails, the disk  log  process
1079              terminates     with     the     EXIT    message    {{failed,Rea‐
1080              son},[{disk_log,Fun,Arity}]}. Other  processes  having  requests
1081              queued    receive   the   message   {disk_log,   Node,   {error,
1082              disk_log_stopped}}.
1083
1084       unblock(Log) -> ok | {error, unblock_error_rsn()}
1085
1086              Types:
1087
1088                 Log = log()
1089                 unblock_error_rsn() =
1090                     no_such_log | nonode |
1091                     {not_blocked, log()} |
1092                     {not_blocked_by_pid, log()}
1093
1094              Unblocks a log. A log can only  be  unblocked  by  the  blocking
1095              process.
1096

SEE ALSO

1098       file(3), pg2(3), wrap_log_reader(3)
1099
1100
1101
1102Ericsson AB                      kernel 6.5.2                      disk_log(3)
Impressum