1ttb(3)                     Erlang Module Definition                     ttb(3)
2
3
4

NAME

6       ttb - A base for building trace tools for distributed systems.
7

DESCRIPTION

9       The  Trace  Tool  Builder,  ttb, is a base for building trace tools for
10       distributed systems.
11
12       When using ttb, do not use module dbg in application  Runtime_Tools  in
13       parallel.
14

EXPORTS

16       start_trace(Nodes, Patterns, FlagSpec, Opts) -> Result
17
18              Types:
19
20                 Result = see p/2
21                 Nodes = see tracer/2
22                 Patterns = [tuple()]
23                 FlagSpec = {Procs, Flags}
24                 Proc = see p/2
25                 Flags = see p/2
26                 Opts = see tracer/2
27
28              This  function  is a shortcut allowing to start a trace with one
29              command. Each tuple in Patterns is converted to a list, which in
30              turn is passed to ttb:tpl/2,3,4.
31
32              The call:
33
34              > ttb:start_trace([Node, OtherNode], [{mod, foo, []}, {mod, bar, 2}], {all, call}, [{file, File}, {handler,{fun myhandler/4, S}}]).
35
36              is equivalent to:
37
38              > ttb:start_trace([Node, OtherNode], [{file, File}, {handler,{fun myhandler/4, S}}]), ttb:tpl(mod, foo, []), ttb:tpl(mod, bar, 2, []), ttb:p(all, call).
39
40       tracer() -> Result
41
42              Equivalent to tracer(node()).
43
44       tracer(Shortcut) -> Result
45
46              Types:
47
48                 Shortcut = shell | dbg
49
50              Handy shortcuts for common tracing settings.
51
52              shell  is  equivalent  to tracer(node(),[{file, {local, "ttb"}},
53              shell]).
54
55              dbg is equivalent to tracer(node(),[{shell, only}]).
56
57       tracer(Nodes) -> Result
58
59              Equivalent to tracer(Nodes,[]).
60
61       tracer(Nodes,Opts) -> Result
62
63              Types:
64
65                 Result = {ok, ActivatedNodes} | {error,Reason}
66                 Nodes = atom() | [atom()] | all | existing | new
67                 Opts = Opt | [Opt]
68                 Opt   =   {file,Client}   |   {handler,   FormatHandler}    |
69                 {process_info,PI}  |  shell  |  {shell,  ShellSpec} | {timer,
70                 TimerSpec} | {overload_check,  {MSec,  Module,  Function}}  |
71                 {flush,   MSec}   |   resume   |   {resume,  FetchTimeout}  |
72                 {queue_size, QueueSize}
73                 TimerSpec = MSec | {MSec, StopOpts}
74                 MSec = FetchTimeout = integer()
75                 Module = Function = atom()
76                 StopOpts = see stop/2
77                 Client = File | {local, File}
78                 File = Filename | Wrap
79                 Filename = string()
80                 Wrap = {wrap,Filename} | {wrap,Filename,Size,Count}
81                 FormatHandler = See format/2
82                 PI = true | false
83                 ShellSpec = true | false | only
84                 QueueSize = non_neg_integer()
85
86              Starts a file trace port on all specified nodes and  points  the
87              system tracer for sequential tracing to the same port.
88
89              Options:
90
91                Filename:
92                  The  specified  Filename  is  prefixed  with  the node name.
93                  Default Filename is ttb.
94
95                File={wrap,Filename,Size,Count}:
96                  Can be used if the size of the trace logs must  be  limited.
97                  Default values are Size=128*1024 and Count=8.
98
99                Client:
100                  When  tracing  diskless  nodes,  ttb must be started from an
101                  external "trace control node" with disk access,  and  Client
102                  must be {local, File}. All trace information is then sent to
103                  the trace control node where it is written to file.
104
105                queue_size:
106                  When tracing to shell or {local,File}, an ip trace driver is
107                  used  internally. The ip trace driver has a queue of maximum
108                  QueueSize messages waiting to be delivered.  If  the  driver
109                  cannot  deliver  messages  as fast as they are produced, the
110                  queue size might be exceeded and messages are dropped.  This
111                  parameter  is  optional, and is only useful if many {drop,N}
112                  trace messages are received by the trace handler. It has  no
113                  meaning   if   shell   or  {local,File}  is  not  used.  See
114                  dbg:trace_port/2 for more information  about  the  ip  trace
115                  driver.
116
117                process_info:
118                  Indicates if process information is to be collected. If PI =
119                  true (which is default),  each  process  identifier  Pid  is
120                  replaced  by  a tuple {Pid,ProcessInfo,Node}, where Process‐
121                  Info is the registered process name, its globally registered
122                  name,  or its initial function. To turn off this functional‐
123                  ity, set PI = false.
124
125                {shell, ShellSpec}:
126                  Indicates that trace messages are to be printed on the  con‐
127                  sole  as  they  are  received  by  the tracing process. This
128                  implies trace client {local, File}.  If  ShellSpec  is  only
129                  (instead of true), no trace logs are stored.
130
131                shell:
132                  Shortcut for {shell, true}.
133
134                timer:
135                  Indicates  that  the  trace  is  to be automatically stopped
136                  after MSec milliseconds.  StopOpts  are  passed  to  command
137                  ttb:stop/2  if  specified  (default  is []). Notice that the
138                  timing is approximate, as delays related to network communi‐
139                  cation are always present. The timer starts after ttb:p/2 is
140                  issued, so you can set up your trace patterns before.
141
142                overload_check:
143                  Allows to enable overload checking on the nodes under trace.
144                  Module:Function(check)  is  performed each MSec millisecond.
145                  If the check returns true, the  tracing  is  disabled  on  a
146                  specified node.
147
148                  Module:Function must be able to handle at least three atoms:
149                  init, check, and stop. init and stop allows you to  initial‐
150                  ize and clean up the check environment.
151
152                  When  a  node  gets  overloaded, it is not possible to issue
153                  ttb:p/2 or any command from the ttb:tp/2,3,4 family,  as  it
154                  would  lead  to  inconsistent tracing state (different trace
155                  specifications on different nodes).
156
157                flush:
158                  Periodically  flushes  all  file  trace  port  clients  (see
159                  dbg:flush_trace_port/1). When enabled, the buffers are freed
160                  each MSec millisecond.  This  option  is  not  allowed  with
161                  {file, {local, File}} tracing.
162
163                {resume, FetchTimeout}:
164                  Enables  the  autoresume feature. When enabled, remote nodes
165                  try to  reconnect  to  the  controlling  node  if  they  are
166                  restarted. The feature requires application Runtime_Tools to
167                  be started (so it has to be present in the .boot scripts  if
168                  the  traced  nodes run with embedded Erlang). If this is not
169                  possible, resume can be performed manually by starting  Run‐
170                  time_Tools remotely using rpc:call/4.
171
172                  ttb  tries to fetch all logs from a reconnecting node before
173                  reinitializing the trace. This must finish within FetchTime‐
174                  out milliseconds or is aborted.
175
176                  By  default, autostart information is stored in a file named
177                  ttb_autostart.bin on each node. If this is not desired  (for
178                  example,  on  diskless  nodes),  a  custom  module  handling
179                  autostart information storage and retrieval can be  provided
180                  by  specifying environment variable ttb_autostart_module for
181                  the application Runtime_Tools. The module  must  respond  to
182                  the following API:
183
184                  write_config(Data) -> ok:
185                    Stores  the  provided  data  for  further retrieval. It is
186                    important to realize that the data storage used  must  not
187                    be affected by the node crash.
188
189                  read_config() -> {ok, Data} | {error, Error}:
190                    Retrieves configuration stored with write_config(Data).
191
192                  delete_config() -> ok:
193                    Deletes   configuration  stored  with  write_config(Data).
194                    Notice that  after  this  call  any  subsequent  calls  to
195                    read_config must return {error, Error}.
196
197                  resume implies the default FetchTimeout, which is 10 seconds
198
199       p(Item,Flags) -> Return
200
201              Types:
202
203                 Return = {ok,[{Item,MatchDesc}]}
204                 Items = Item | [Item]
205                 Item  =  pid()  | port() | RegName | {global,GlobalRegName} |
206                 all | processes | ports |  existing  |  existing_processes  |
207                 existing_ports | new | new_processes | new_ports
208                 RegName = atom()
209                 GlobalRegName = term()
210                 Flags = Flag | [Flag]
211
212              Sets  the  specified  trace  flags on the specified processes or
213              ports. Flag timestamp is always turned on.
214
215              See the Reference Manual for module dbg for the  possible  trace
216              flags. Parameter MatchDesc is the same as returned from dbg:p/2.
217
218              Processes  can be specified as registered names, globally regis‐
219              tered names, or process identifiers. Ports can be  specified  as
220              registered  names  or  port identifiers. If a registered name is
221              specified, the flags are set on processes/ports with  this  name
222              on all active nodes.
223
224              Issuing  this  command starts the timer for this trace if option
225              timer is specified with tracer/2.
226
227       tp, tpl, tpe, ctp, ctpl, ctpg, ctpe
228
229              These functions are to be used with trace flag call,  send,  and
230              'receive' for setting and clearing trace patterns.
231
232              When  trace  flag  call  is set on a process, function calls are
233              traced on that process if a trace pattern is set for the  called
234              function.
235
236              The send and 'receive' flags enable tracing of all messages sent
237              and received by the process/port. Trace patterns  set  with  tpe
238              may  limit  traced  messages  based  on the message content, the
239              sender, and/or the receiver.
240
241              Trace patterns specify how to trace a function or a  message  by
242              using  match  specifications. Match specifications are described
243              in the ERTS User's Guide.
244
245              These functions are equivalent to the corresponding functions in
246              module dbg, but all calls are stored in the history. The history
247              buffer makes it easy to create  configuration  files;  the  same
248              trace environment can be set up many times, for example, to com‐
249              pare two test runs. It also reduces the amount  of  typing  when
250              using ttb from the Erlang shell.
251
252                tp:
253                  Sets trace patterns on global function calls.
254
255                tpl:
256                  Sets trace patterns on local and global function calls.
257
258                tpe:
259                  Sets trace patterns on messages.
260
261                ctp:
262                  Clears trace patterns on local and global function calls.
263
264                ctpl:
265                  Clears trace patterns on local function calls.
266
267                ctpg:
268                  Clears trace patterns on global function calls.
269
270                ctpe:
271                  Clears trace patterns on messages.
272
273              With tp and tpl, one of the match specification shortcuts can be
274              used (for example, ttb:tp(foo_module, caller)).
275
276              The shortcuts are as follows:
277
278                * return - for [{'_',[],[{return_trace}]}] (report the  return
279                  value from a traced function)
280
281                * caller  -  for  [{'_',[],[{message,{caller}}]}]  (report the
282                  calling function)
283
284                * {codestr,  Str}  -  for  dbg:fun2ms/1  arguments  passed  as
285                  strings (example: "fun(_) -> return_trace() end")
286
287       list_history() -> History
288
289              Types:
290
291                 History = [{N,Func,Args}]
292
293              All calls to ttb is stored in the history. This function returns
294              the current content of the history. Any entry can be  reexecuted
295              with  run_history/1  or  stored  in  a  configuration  file with
296              write_config/2,3.
297
298       run_history(N) -> ok | {error, Reason}
299
300              Types:
301
302                 N = integer() | [integer()]
303
304              Executes the specified entry or entries from the  history  list.
305              To list history, use list_history/0.
306
307       write_config(ConfigFile,Config)
308
309              Equivalent to write_config(ConfigFile,Config,[]).
310
311       write_config(ConfigFile,Config,Opts) -> ok | {error,Reason}
312
313              Types:
314
315                 ConfigFile = string()
316                 Config = all | [integer()] | [{Mod,Func,Args}]
317                 Mod = atom()
318                 Func = atom()
319                 Args = [term()]
320                 Opts = Opt | [Opt]
321                 Opt = append
322
323              Creates  or  extends a configuration file, which can be used for
324              restoring a specific configuration later.
325
326              The contents of the configuration file  can  either  be  fetched
327              from   the   history   or   specified  directly  as  a  list  of
328              {Mod,Func,Args}.
329
330              If the complete history is to be  stored  in  the  configuration
331              file,  Config  must be all. If only a selected number of entries
332              from the history are to be stored, Config  must  be  a  list  of
333              integers pointing out the entries to be stored.
334
335              If  Opts  is not specified or if it is [], ConfigFile is deleted
336              and a new file is created. If Opts = [append], ConfigFile is not
337              deleted. The new information is appended at the end of the file.
338
339       run_config(ConfigFile) -> ok | {error,Reason}
340
341              Types:
342
343                 ConfigFile = string()
344
345              Executes all entries in the specified configuration file. Notice
346              that the history of the last trace is always available  in  file
347              ttb_last_config.
348
349       run_config(ConfigFile,NumList) -> ok | {error,Reason}
350
351              Types:
352
353                 ConfigFile = string()
354                 NumList = [integer()]
355
356              Executes selected entries from the specified configuration file.
357              NumList is a list of integers pointing out  the  entries  to  be
358              executed.
359
360              To list the contents of a configuration file, use list_config/1.
361
362              Notice that the history of the last trace is always available in
363              file ttb_last_config.
364
365       list_config(ConfigFile) -> Config | {error,Reason}
366
367              Types:
368
369                 ConfigFile = string()
370                 Config = [{N,Func,Args}]
371
372              Lists all entries in the specified configuration file.
373
374       write_trace_info(Key,Info) -> ok
375
376              Types:
377
378                 Key = term()
379                 Info = Data | fun() -> Data
380                 Data = term()
381
382              File .ti contains {Key,ValueList}  tuples.  This  function  adds
383              Data to the ValueList associated with Key. All information writ‐
384              ten with this function is included in the  call  to  the  format
385              handler.
386
387       seq_trigger_ms() -> MatchSpec
388
389              Equivalent to seq_trigger_ms(all).
390
391       seq_trigger_ms(Flags) -> MatchSpec
392
393              Types:
394
395                 MatchSpec = match_spec()
396                 Flags = all | SeqTraceFlag | [SeqTraceFlag]
397                 SeqTraceFlag = atom()
398
399              A  match  specification  can  turn on or off sequential tracing.
400              This function returns a  match  specification,  which  turns  on
401              sequential tracing with the specified Flags.
402
403              This  match  specification can be specified as the last argument
404              to tp or tpl. The activated Item  then  becomes  a  trigger  for
405              sequential  tracing.  This means that if the item is called on a
406              process with trace flag call set, the process is  "contaminated"
407              with token seq_trace.
408
409              If Flags = all, all possible flags are set.
410
411              The possible values for SeqTraceFlag are available in seq_trace.
412
413              For  a description of the match_spec() syntax, see section Match
414              Specifications in Erlang in ERTS,  which  explains  the  general
415              match specification "language".
416
417          Note:
418              The system tracer for sequential tracing is automatically initi‐
419              ated by ttb when a trace port is started with ttb:tracer/0,1,2.
420
421
422              An example of how to use function seq_trigger_ms/0,1 follows:
423
424              (tiger@durin)5> ttb:tracer().
425              {ok,[tiger@durin]}
426              (tiger@durin)6> ttb:p(all,call).
427              {ok,{[all],[call]}}
428              (tiger@durin)7> ttb:tp(mod,func,ttb:seq_trigger_ms()).
429              {ok,[{matched,1},{saved,1}]}
430              (tiger@durin)8>
431
432              Whenever mod:func(...) is called after this, token seq_trace  is
433              set on the executing process.
434
435       stop()
436
437              Equivalent to stop([]).
438
439       stop(Opts) -> stopped | {stopped, Dir}
440
441              Types:
442
443                 Opts = Opt | [Opt]
444                 Opt  =  nofetch  |  {fetch_dir, Dir} | format | {format, For‐
445                 matOpts} | return_fetch_dir
446                 Dir = string()
447                 FormatOpts = see format/2
448
449              Stops tracing on all nodes. Logs and trace information files are
450              sent  to  the trace control node and stored in a directory named
451              ttb_upload_FileName-Timestamp, where Filename is  the  one  pro‐
452              vided  with  {file, File} during trace setup and Timestamp is of
453              the form yyyymmdd-hhmmss. Even  logs  from  nodes  on  the  same
454              machine  as  the trace control node are moved to this directory.
455              The history list is saved to a file  named  ttb_last_config  for
456              further reference (as it is no longer accessible through history
457              and  configuration  management  functions,  like   ttb:list_his‐
458              tory/0).
459
460              Options:
461
462                nofetch:
463                  Indicates  that  trace  logs  are  not to be collected after
464                  tracing is stopped.
465
466                {fetch, Dir}:
467                  Allows specification of the directory to fetch the data  to.
468                  If the directory already exists, an error is thrown.
469
470                format:
471                  Indicates  the  trace  logs to be formatted after tracing is
472                  stopped. All logs in the fetch directory are merged.
473
474                return_fetch_dir:
475                  Indicates the return value to be {stopped, Dir} and not just
476                  stopped. This implies fetch.
477
478       get_et_handler()
479
480              Returns  the  et  handler,  which  can  be used with format/2 or
481              tracer/2.
482
483              Example: ttb:format(Dir, [{handler, ttb:get_et_handler()}]).
484
485       format(File)
486
487              Equivalent to format(File,[]).
488
489       format(File,Options) -> ok | {error, Reason}
490
491              Types:
492
493                 File = string() | [string()]
494                   This can be the name of a binary log, a list of such  logs,
495                   or  the  name  of a directory containing one or more binary
496                   logs.
497                 Options = Opt | [Opt]
498                 Opt = {out,Out} | {handler,FormatHandler} | disable_sort
499                 Out = standard_io | string()
500                 FormatHandler = {Function, InitialState}
501                 Function = fun(Fd,Trace,TraceInfo,State) -> State
502                 Fd = standard_io | FileDescriptor
503                   File descriptor of the destination file Out.
504                 Trace = tuple()
505                   The trace message. For details, see  the  Reference  Manual
506                   for module erlang.
507                 TraceInfo = [{Key,ValueList}]
508                   Includes  the  keys  flags, client, and node. If handler is
509                   specified as option to the tracer function,  this  is  also
510                   included.  Also,  all  information  written  with  function
511                   write_trace_info/2 is included.
512
513              Reads the specified binary trace log(s). The logs are  processed
514              in the order of their time stamps as long as option disable_sort
515              is not specified.
516
517              If FormatHandler = {Function,InitialState}, Function  is  called
518              for each trace message.
519
520              If FormatHandler = get_et_handler(), et_viewer in application ET
521              is used for presenting the trace log graphically. ttb provides a
522              few different filters that can be selected from menu Filters and
523              scaling in the et_viewer.
524
525              If FormatHandler is not specified, a  default  handler  is  used
526              presenting each trace message as a text line.
527
528              The  state  returned from each call of Function is passed to the
529              next call, even if the next call is to  format  a  message  from
530              another log file.
531
532              If  Out  is specified, FormatHandler gets the file descriptor to
533              Out as the first parameter.
534
535              Out is ignored if the et format handler is used.
536
537              Wrap logs can be formatted one by one or all at once. To  format
538              one  of  the wrap logs in a set, specify the exact file name. To
539              format the whole set of wrap  logs,  specify  the  name  with  *
540              instead of the wrap count. For examples, see the User's Guide.
541
542
543
544Ericsson AB                      observer 2.7                           ttb(3)
Impressum