1FY-TOOL(1) libfyaml FY-TOOL(1)
2
6 fy-tool - fy-tool documentation
7
9 fy-tool [OPTIONS] [<file> ...]
10 fy-dump [OPTIONS] [<file> ...]
12 fy-testsuite [OPTIONS] <file>
14 fy-filter [OPTIONS] [-f*FILE*] [<path> ...]
16 fy-join [OPTIONS] [-T*PATH*] [-F*PATH*] [-t*PATH*] [<file> ...]
18 fy-ypath [OPTIONS] <ypath-expression> [<*file> ...]
20 fy-compose [OPTIONS] [<file> ...]
22
24 fy-tool is a general YAML/JSON manipulation tool using libfyaml. Its
25 operation is different depending on how it's called, either via aliases
26 named fy-dump, fy-testsuite, fy-filter, fy-join, or via the --dump,
27 --testsuite, --filter, --join, --ypath and --compose options.
28 • In dump mode it will parse YAML/JSON input files and output YAML/JSON
30 according to the output format options.
31 • In testsuite mode it will parse a single YAML input file and output
33 yaml testsuite reference output.
34 • In filter mode it will parse YAML/JSON files and output YAML/JSON
36 output filtered according to the given option.
37 • In join mode it will parse YAML/JSON files and join them into a sin‐
39 gle YAML/JSON document according to the given command line options.
40 • In ypath mode it will parse YAML/JSON files and execute a ypath query
42 which will output a document stream according to the results. This is
43 an experimental mode under development, where the syntax is not yet
44 decided completely.
45 • In compose mode, it operates similarly to dump, but the document tree
47 is created using the built-in composer API.
48
50 A number of options are common for all the different modes of operation
51 and they are as follows: Common options.INDENT 0.0
52 -q, --quiet
54 Quiet operation, does not output informational messages at all.
55 -h, --help
57 Display usage information.
58 -v, --version
60 Display version information.
61 -I DIR, --include=DIR
63 Add the DIR directory to the search path which will be used to
64 locate a YAML/JSON file. The default path is set to ""
65 -d LEVEL, --debug-level=LEVEL
67 Set the minimum numeric debug level value of the library to
68 LEVEL. The numeric value must be in the range of 0 to 4 and
69 their meaning is as follows:
70 • 0 (DEBUG)
72 Internal library debugging messages. No output is produced
74 when the library was compiled with --disable-debug
75 • 1 (INFO)
77 Informational messages about the internal operation of the li‐
79 brary.
80 • 2 (NOTICE)
82 Messsages that could require attention.
84 • 3 (WARNING)
86 A warning message, something is wrong, but operation can con‐
88 tinue. This is the default value.
89 • 4 (ERROR)
91 A fatal error occured, it is impossible to continue.
93 The default level is 3 (WARNING), which means that messages with
95 level 3 and higher will be displayed.
96 Parser Options.INDENT 0.0
97 -j JSONOPT, --json=JSONOPT
99 Marks the input files as JSON or YAML accordingly to:
100 • no
102 The input files are always in YAML mode.
104 • force
106 The input files are always set to JSON mode.
108 • auto
110 The input files are set to JSON mode automatically when the
112 file's extension is .json. This is the default.
113 JSON support is complete so all valid JSON files are parsed ac‐
115 cording to JSON rules, even where those differ with YAML rules.
116 --yaml-1.1
118 Force YAML 1.1 rules, when input does not specify a version via
119 a directive.
120 --yaml-1.2
122 Force YAML 1.2 rules, when input does not specify a version via
123 a directive.
124 --yaml-1.3
126 Force YAML 1.3 rules, when input does not specify a version via
127 a directive. This option is experimental since the 1.3 spec is
128 not yet released.
129 --disable-accel
131 Disable use of acceleration features; use less memory but poten‐
132 tially more CPU.
133 --disable-buffering
135 Disable use stdio bufferring, reads will be performed via unix
136 fd reads. This may reduce latency when reading from a network
137 file descriptor, or similar.
138 --disable-depth-limit
140 Disable the object depth limit, which is usually set to a value
141 near 60. Using this option is is possible to process even
142 pathological inputs when using the default non-recursive build
143 mode.
144 --prefer-recursive
146 Prefer recursive build methods, instead of iterative. This field
147 is merely here for evaluation purposes and will be removed in a
148 future version.
149 --sloppy-flow-indentation
151 Use sloppy flow indentation, where indentation is not taken into
152 account in flow mode, even when the input is invalid YAML ac‐
153 cording to the spec.
154 --ypath-aliases
156 Process aliases using ypaths. Experimental option.
157 --streaming
159 Only valid when in dump mode, enables streaming mode. This means
160 that no in-memory graph tree is constructed, so indefinite and
161 arbitrary large YAML input streams can be processed.
162 Note that in streaming mode:
164 • Key duplication checks are disabled.
166 • No reording of key order is possible when emitting (i.e.
168 --sort is not available).
169 • Alias resolution is not available (i.e. --resolve).
171 Resolver Options.INDENT 0.0
172 -r, --resolve
174 Perform anchor and merge key resolution. By default this option
175 is disabled.
176 -l, --follow
178 Follow aliases when performing path traversal. By default this
179 option is disabled.
180 Testsuite Options.INDENT 0.0
181 --disable-flow-markers
183 Do not output flow-markers for the testsuite output.
184Emitter Options.INDENT 0.0
185 -i INDENT, --indent=INDENT
187 Sets the emitter indent (in spaces). Default is 2.
188 -w WIDTH, --width=WIDTH
190 Sets the preferred output width of the emitter. It is generally
191 impossible to strictly adhere to this limit so this is treated
192 as a hint at best. It not valid in any oneline output modes
193 (i.e. flow-oneline or json-oneline). Default value is 80.
194 -m MODE, --mode=MODE
196 Sets the output mode of the YAML emitted. Possible values are:
197 • original
199 The original formatting used in the input. This is default
201 mode.
202 • block
204 The output is forced to be in block mode. All flow constructs
206 will be converted to block mode.
207 • flow
209 The output is forced to be in flow mode. All block constructs
211 will be converted to flow mode.
212 • flow-oneline
214 The output is forced to be in flow mode, but no newlines will
216 be emitted; the output is going to be a (potentially very)
217 long line.
218 • json
220 The output is forced to be in JSON mode. Note that it is im‐
222 possible to output an arbitrary YAML file as JSON, so this may
223 fail.
224 • json-oneline
226 The output is forced to be in JSON mode and in a single line.
228 • dejson
230 Output is in block YAML mode but with special care to convert
232 JSON quoted strings in as non-idiomatic YAML as possible. For
233 example { foo: "this is a test" } will be emitted as foo: this
234 is a test. YAML can handle scalars without using excessive
235 quoting.
236 -C MODE, --color=MODE
238 It is possible to colorize output using ANSI color escape se‐
239 quences, and the mode can be one of:
240 • off
242 Never colorize output.
244 • on
246 Always colorize output.
248 • auto
250 Automatically colorize output when the output is a terminal.
252 This is the default.
253 -V, --visible
255 Make all whitespace (spaces, unicode spaces and linebreaks) vis‐
256 ible. Note that this is performed using UTF8 characters so it
257 will not work on non-UTF8 terminals, or a non-UTF8 complete
258 font.
259 -s, --sort
261 Sort keys on output. This option is disabled by default.
262 -c, --comment
264 Experimental output comments option. Enabled output while pre‐
265 serving comments. Disabled by default.
266 --strip-labels
268 Strip labels on output. Disabled by default.
269 --strip-tags
271 Strip tags on output. Disabled by default.
272 --strip-doc
274 Strip document indicators on output. Disabled by default.
275 --null-output
277 Do not generate any output, useful for profiling the parser.
278 YPATH options.INDENT 0.0
279 --dump-pathexpr
281 Dump the produced path expression for debugging.
282 --no-exec
284 Do not execute the expression. Useful when used with
285 --dump-pathexpr
286 Compose options.INDENT 0.0
287 --dump-path
289 Dump the path while composing.
290Tool mode select options.INDENT 0.0
291 --dump Select dump mode of operation. This is the default. This mode
293 is also enabled when the called binary is aliased to fy-dump.
294 In this mode, all files provided in the command line will be
296 dumped in one continuous stream, to the standard output, using
297 document start indicators to mark the start of end new file.
298 If the file provided is - then the input is the standard input.
300 --testsuite
302 Select testsuite mode of operation. This mode is also enabled
303 when the called binary is aliased to fy-testsuite.
304 In this mode, a single YAML file is read and an event stream is
306 generated which is the format used for yaml-testsuite compli‐
307 ance.
308 If the file provided is - then the input is the standard input.
310 --filter
312 Select filter mode of operation. This mode is also enabled when
313 the called binary is aliased to fy-filter.
314 In this mode, a single YAML file is read from the standard input
316 for each path that is provided in the command line a document
317 will be produced to the standard output. To use file instead of
318 standard input use the -f/--file option.
319 If the file provided is - then the input is the standard input.
321 -f FILE, --file=FILE
323 Use the given file as input instead of standard input.
324 If first character of FILE is > the the input is the
326 content of the option that follows. For example --file
327 ">foo: bar" is as --file file.yaml with file.yaml "foo:
328 bar"
329 --join Select join mode of operation. This mode is also enabled when
331 the called binary is aliased to fy-join.
332 In this mode, multiple YAML files are joined into a single docu‐
334 ment, emitted to the standard output.
335 If the file provided is - then the input is the standard input.
337 -T PATH, --to=PATH
339 The target path of the join. By default this is the root
340 /.
341 If first character of FILE is > the the input is the
343 content of the option that follows.
344 -F PATH, --from=PATH
346 The origin path of the join (for each input). By default
347 this is the root /.
348 If first character of FILE is > the the input is the
350 content of the option that follows.
351 -t PATH, --trim=PATH
353 Trim path of the output of the join. By default this is
354 the root /.
355 If first character of FILE is > the the input is the
357 content of the option that follows.
358 --ypath
360 Process files and output query results using ypath.
361 --compose
363 Use the composer API to build the document instead of direct
364 events.
365
367 Example input files
368 We're going to be using a couple of YAML files in our examples.
370 invoice.yaml
372 # invoice.yaml
374 invoice: 34843
375 date : !!str 2001-01-23
376 bill-to: &id001
377 given : Chris
378 family : Dumars
379 address:
380 lines: |
381 458 Walkman Dr.
382 Suite #292
383 simple-anchors.yaml
385 # simple-anchors.yaml
387 foo: &label { bar: frooz }
388 baz: *label
389 mergekeyspec.yaml
391 ---
393 - &CENTER { x: 1, y: 2 }
394 - &LEFT { x: 0, y: 2 }
395 - &BIG { r: 10 }
396 - &SMALL { r: 1 }
397 # All the following maps are equal:
399 - # Explicit keys
401 x: 1
402 y: 2
403 r: 10
404 label: center/big
405 - # Merge one map
407 << : *CENTER
408 r: 10
409 label: center/big
410 - # Merge multiple maps
412 << : [ *CENTER, *BIG ]
413 label: center/big
414 - # Override
416 << : [ *BIG, *LEFT, *SMALL ]
417 x: 1
418 label: center/big
419 bomb.yaml
421 a: &a ["lol","lol","lol","lol","lol","lol","lol","lol","lol"]
423 b: &b [*a,*a,*a,*a,*a,*a,*a,*a,*a]
424 c: &c [*b,*b,*b,*b,*b,*b,*b,*b,*b]
425 d: &d [*c,*c,*c,*c,*c,*c,*c,*c,*c]
426 e: &e [*d,*d,*d,*d,*d,*d,*d,*d,*d]
427 f: &f [*e,*e,*e,*e,*e,*e,*e,*e,*e]
428 g: &g [*f,*f,*f,*f,*f,*f,*f,*f,*f]
429 fy-dump examples.
430 Parse and dump generated YAML document tree in the original YAML form
432 $ fy-dump invoice.yaml
434 invoice: 34843
436 date: !!str 2001-01-23
437 bill-to: &id001
438 given: Chris
439 family: Dumars
440 address:
441 lines: |
442 458 Walkman Dr.
443 Suite #292
444 Parse and dump generated YAML document tree in flow YAML form
446 $ fy-dump -mflow invoice.yaml
448 {
450 invoice: 34843,
451 date: !!str 2001-01-23,
452 bill-to: &id001 {
453 given: Chris,
454 family: Dumars,
455 address: {
456 lines: "458 Walkman Dr.\nSuite #292\n"
457 }
458 }
459 }
460 Parse and dump generated YAML document from the input string
462 $ fy-dump -mjson ">foo: bar"
464 {
466 "foo": "bar"
467 }
468 Using the resolve option on the simple-anchors.yaml
470 $ fy-dump -r simple-anchor.yaml
472 foo: &label {
474 bar: frooz
475 }
476 baz: {
477 bar: frooz
478 }
479 Stripping the labels too:
481 $ fy-dump -r --strip-label simple-anchor.yaml
483 foo: {
485 bar: frooz
486 }
487 baz: {
488 bar: frooz
489 }
490 Merge key support:
492 $ fy-dump -r --strip-label mergekeyspec.yaml
494 ---
496 - {
497 x: 1,
498 y: 2
499 }
500 - {
501 x: 0,
502 y: 2
503 }
504 - {
505 r: 10
506 }
507 - {
508 r: 1
509 }
510 - x: 1
511 y: 2
512 r: 10
513 label: center/big
514 - y: 2
515 x: 1
516 r: 10
517 label: center/big
518 - r: 10
519 y: 2
520 x: 1
521 label: center/big
522 - y: 2
523 r: 10
524 x: 1
525 label: center/big
526 Sorting option:
528 $ fy-dump -s invoice.yaml
530 bill-to: &id001
532 address:
533 lines: |
534 458 Walkman Dr.
535 Suite #292
536 family: Dumars
537 given: Chris
538 date: !!str 2001-01-23
539 invoice: 34843
540 fy-testsuite example.
541 An example using the testsuite mode generates the following event
543 stream from invoice.yaml
544 Parse and dump test-suite event format
546 $ fy-testsuite invoice.yaml
548 fy-filter examples.
549 Filter out from the /bill-to path of invoice.yaml
551 $ cat invoice.yaml | fy-filter /bill-to
553 &id001
555 given: Chris
556 family: Dumars
557 address:
558 lines: |
559 458 Walkman Dr.
560 Suite #292
561 Filter example with arrays (and use the --file option)
563 $ fy-filter --file=mergekeyspec.yaml /5
565 ---
567 <<: *CENTER
568 r: 10
569 label: center/big
570 Follow anchors example
572 $ fy-filter --file=simple-anchors.yaml /baz/bar
574 frooz
576 Handle YAML bombs (if you can spare the memory and cpu time)
578 $ fy-filter --file=bomb.yaml -r / | wc -l
580 6726047
581 You don't have to, you can just follow links to retrieve data.
583 $ fy-filter --file=stuff/bomb.yaml -l --strip-label /g/0/1/2/3/4/5/6
585 "lol"
587 Following links works with merge keys too:
589 $ fy-filter --file=mergekeyspec.yaml -l --strip-label /5/x
591 --- 1
593 fy-join examples.
594 Joining two YAML files that have root mappings.
596 $ fy-join simple-anchors.yaml invoice.yaml
598 foo: &label {
600 bar: frooz
601 }
602 baz: *label
603 invoice: 34843
604 date: !!str 2001-01-23
605 bill-to: &id001
606 given: Chris
607 family: Dumars
608 address:
609 lines: |
610 458 Walkman Dr.
611 Suite #292
612 Join two files with sequences at root:
614 $ fy-join -mblock ">[ foo, bar ]" ">[ baz ]"
616 - foo
618 - bar
619 - baz
620
622 Pantelis Antoniou <pantelis.antoniou@konsulko.com>
623
625 • The only supported input and output character encoding is UTF8.
626 • Sorting does not respect language settings.
628 • There is no way for the user to specific a different coloring scheme.
630
632 libfyaml(1)
633
635 2019, Pantelis Antoniou
636 Jan 14, 2022 FY-TOOL(1)