1NGRAMSYMBOLS(1)                  User Commands                 NGRAMSYMBOLS(1)
2
3
4

NAME

6       ngramsymbols - manual page for ngramsymbols 1.3.14
7

DESCRIPTION

9       Derives a symbol table from a corpus.
10
11              Usage: ngramsymbols [--options] [in.txt [out.txt]]
12
13       PROGRAM FLAGS:
14
15       --OOV_symbol: type = std::string, default = "<UNK>"
16
17              Class   label   for   OOV   symbols   --epsilon_symbol:  type  =
18              std::string, default = "<epsilon>" Label for epsilon
19
20       LIBRARY FLAGS:
21
22       Flags from: flags.cc
23
24       --help: type = bool, default = false
25
26              show usage information --helpshort: type = bool, default = false
27              show  brief  usage information --tmpdir: type = std::string, de‐
28              fault = "/tmp" temporary directory --v: type = int32_t,  default
29              = 0 verbosity level
30
31       Flags from: fst.cc
32
33       --fst_align: type = bool, default = false
34
35              Write FST data aligned where appropriate --fst_default_cache_gc:
36              type = bool, default = true Enable garbage collection  of  cache
37              --fst_default_cache_gc_limit:  type = int64_t, default = 1048576
38              Cache   byte   size    that    triggers    garbage    collection
39              --fst_read_mode:  type  =  std::string, default = "read" Default
40              file reading mode for  mappable  files  --fst_verify_properties:
41              type  =  bool,  default = false Verify FST properties queried by
42              TestProperties --save_relabel_ipairs: type  =  std::string,  de‐
43              fault  =  ""  Save  input  relabel  pairs  to  file --save_rela‐
44              bel_opairs: type = std::string, default = "" Save output relabel
45              pairs to file
46
47       Flags from: ngram-output.cc
48
49       --end_symbol: type = std::string, default = "</s>"
50
51              Class label for sentence end --start_symbol: type = std::string,
52              default = "<s>" Class label for sentence start
53
54       Flags from: symbol-table.cc
55
56       --fst_compat_symbols: type = bool, default = true
57
58              Require symbol tables to match when appropriate --fst_field_sep‐
59              arator: type = std::string, default = "         " Set of charac‐
60              ters used as a separator between printed fields
61
62       Flags from: util.cc
63
64       --fst_error_fatal: type = bool, default = true
65
66              FST errors are fatal; o.w. return objects flagged as bad:  e.g.,
67              FSTs:   kError   property  set,  FST  weights:  not  a  Member()
68              --ngram_error_fatal: type = bool, default =  true  NGram  errors
69              are  fatal  if  true;  otherwise returns objects flagged as bad:
70              e.g., NGramModel::Error() is true
71
72       Flags from: weight.cc
73
74       --fst_weight_parentheses: type = std::string, default = ""
75
76              Characters enclosing the first weight  of  a  printed  composite
77              weight  (e.g., pair weight, tuple weight and derived classes) to
78              ensure proper I/O of nested composite weights; must have size  0
79              (none) or 2 (open and close parenthesis) --fst_weight_separator:
80              type = std::string, default = ","  Character  separator  between
81              printed composite weights; must be a single character
82
83
84
85ngramsymbols 1.3.14              February 2022                 NGRAMSYMBOLS(1)
Impressum