1YAML::PP(3) User Contributed Perl Documentation YAML::PP(3)
2
6 YAML::PP - YAML 1.2 processor
7
9 WARNING: Most of the inner API is not stable yet.
10 Here are a few examples of the basic load and dump methods:
12 use YAML::PP;
14 my $ypp = YAML::PP->new;
15 my $yaml = <<'EOM';
17 --- # Document one is a mapping
18 name: Tina
19 age: 29
20 favourite language: Perl
21 --- # Document two is a sequence
23 - plain string
24 - 'in single quotes'
25 - "in double quotes we have escapes! like \t and \n"
26 - | # a literal block scalar
27 line1
28 line2
29 - > # a folded block scalar
30 this is all one
31 single line because the
32 linebreaks will be folded
33 EOM
34 my @documents = $ypp->load_string($yaml);
36 my @documents = $ypp->load_file($filename);
37 my $yaml = $ypp->dump_string($data1, $data2);
39 $ypp->dump_file($filename, $data1, $data2);
40 # The loader offers JSON::PP::Boolean, boolean.pm or
42 # perl 1/'' (currently default) for booleans
43 my $ypp = YAML::PP->new(boolean => 'JSON::PP');
44 my $ypp = YAML::PP->new(boolean => 'boolean');
45 my $ypp = YAML::PP->new(boolean => 'perl');
46 # Enable perl data types and objects
48 my $ypp = YAML::PP->new(schema => [qw/ + Perl /]);
49 my $yaml = $yp->dump_string($data_with_perl_objects);
50 # Legacy interface
52 use YAML::PP qw/ Load Dump LoadFile DumpFile /;
53 my @documents = Load($yaml);
54 my @documents = LoadFile($filename);
55 my @documents = LoadFile($filehandle);
56 my $yaml = = Dump(@documents);
57 DumpFile($filename, @documents);
58 DumpFile($filenhandle @documents);
59 Some utility scripts, mostly useful for debugging:
61 # Load YAML into a data structure and dump with Data::Dumper
63 yamlpp-load < file.yaml
64 # Load and Dump
66 yamlpp-load-dump < file.yaml
67 # Print the events from the parser in yaml-test-suite format
69 yamlpp-events < file.yaml
70 # Parse and emit events directly without loading
72 yamlpp-parse-emit < file.yaml
73 # Create ANSI colored YAML. Can also be useful for invalid YAML, showing
75 # you the exact location of the error
76 yamlpp-highlight < file.yaml
77
79 YAML::PP is a modular YAML processor.
80 It aims to support "YAML 1.2" and "YAML 1.1". See <https://yaml.org/>.
82 Some (rare) syntax elements are not yet supported and documented below.
83 YAML is a serialization language. The YAML input is called "YAML
85 Stream". A stream consists of one or more "Documents", separated by a
86 line with a document start marker "---". A document optionally ends
87 with the document end marker "...".
88 This allows one to process continuous streams additionally to a fixed
90 input file or string.
91 The YAML::PP frontend will currently load all documents, and return
93 only the first if called with scalar context.
94 The YAML backend is implemented in a modular way that allows one to add
96 custom handling of YAML tags, perl objects and data types. The inner
97 API is not yet stable. Suggestions welcome.
98 You can check out all current parse and load results from the yaml-
100 test-suite here:
101 <https://perlpunk.github.io/YAML-PP-p5/test-suite.html>
102
104 new
105 my $ypp = YAML::PP->new;
106 # load booleans via boolean.pm
107 my $ypp = YAML::PP->new( boolean => 'boolean' );
108 # load booleans via JSON::PP::true/false
109 my $ypp = YAML::PP->new( boolean => 'JSON::PP' );
110 # use YAML 1.2 Failsafe Schema
112 my $ypp = YAML::PP->new( schema => ['Failsafe'] );
113 # use YAML 1.2 JSON Schema
114 my $ypp = YAML::PP->new( schema => ['JSON'] );
115 # use YAML 1.2 Core Schema
116 my $ypp = YAML::PP->new( schema => ['Core'] );
117 # Die when detecting cyclic references
119 my $ypp = YAML::PP->new( cyclic_refs => 'fatal' );
120 my $ypp = YAML::PP->new(
122 boolean => 'JSON::PP',
123 schema => ['Core'],
124 cyclic_refs => 'fatal',
125 indent => 4,
126 header => 1,
127 footer => 1,
128 version_directive => 1,
129 );
130 Options:
132 boolean
134 Values: "perl" (currently default), "JSON::PP", "boolean"
135 This option is for loading and dumping.
137 Note that when dumping, only the chosen boolean style will be
139 recognized. So if you choose "JSON::PP", "boolean" objects will
140 not be recognized as booleans and will be dumped as ordinary
141 objects (if you enable the Perl schema).
142 schema
144 Default: "['Core']"
145 This option is for loading and dumping.
147 Array reference. Here you can define what schema to use. Supported
149 standard Schemas are: "Failsafe", "JSON", "Core", "YAML1_1".
150 To get an overview how the different Schemas behave, see
152 <https://perlpunk.github.io/YAML-PP-p5/schemas.html>
153 Additionally you can add further schemas, for example "Merge".
155 cyclic_refs
157 Default: 'allow' but will be switched to fatal in the future for
158 safety!
159 This option is for loading only.
161 Defines what to do when a cyclic reference is detected when
163 loading.
164 # fatal - die
166 # warn - Just warn about them and replace with undef
167 # ignore - replace with undef
168 # allow - Default
169 duplicate_keys
171 Default: 1
172 Since version 0.026
174 This option is for loading.
176 NOTE: THIS OPTION WILL BE SET TO 0 IN THE NEXT RELEASE.
178 The YAML Spec says duplicate mapping keys should be forbidden.
180 When set to true, duplicate keys in mappings are allowed (and will
182 overwrite the previous key).
183 When set to false, duplicate keys will result in an error when
185 loading.
186 This is especially useful when you have a longer mapping and don't
188 see the duplicate key in your editor:
189 ---
191 a: 1
192 b: 2
193 # .............
194 a: 23 # error
195 indent
197 Default: 2
198 This option is for dumping.
200 Use that many spaces for indenting
202 width
204 Since version 0.025
205 Default: 80
207 This option is for dumping.
209 Maximum columns when dumping.
211 This is only respected when dumping flow collections right now.
213 in the future it will be used also for wrapping long strings.
215 header
217 Default: 1
218 This option is for dumping.
220 Print document heaader "---"
222 footer
224 Default: 0
225 This option is for dumping.
227 Print document footer "..."
229 yaml_version
231 Since version 0.020
232 This option is for loading and dumping.
234 Default: 1.2
236 Note that in this case, a directive "%YAML 1.1" will basically be
238 ignored and everything loaded with the "1.2 Core" Schema.
239 If you want to support both YAML 1.1 and 1.2, you have to specify
241 that, and the schema ("Core" or "YAML1_1") will be chosen
242 automatically.
243 my $yp = YAML::PP->new(
245 yaml_version => ['1.2', '1.1'],
246 );
247 This is the same as
249 my $yp = YAML::PP->new(
251 schema => ['+'],
252 yaml_version => ['1.2', '1.1'],
253 );
254 because the "+" stands for the default schema per version.
256 When loading, and there is no %YAML directive, 1.2 will be
258 considered as default, and the "Core" schema will be used.
259 If there is a "%YAML 1.1" directive, the "YAML1_1" schema will be
261 used.
262 Of course, you can also make 1.1 the default:
264 my $yp = YAML::PP->new(
266 yaml_version => ['1.1', '1.2'],
267 );
268 You can also specify 1.1 only:
270 my $yp = YAML::PP->new(
272 yaml_version => ['1.1'],
273 );
274 In this case also documents with "%YAML 1.2" will be loaded with
276 the "YAML1_1" schema.
277 version_directive
279 Since version 0.020
280 This option is for dumping.
282 Default: 0
284 Print Version Directive "%YAML 1.2" (or "%YAML 1.1") on top of each
286 YAML document. It will use the first version specified in the
287 "yaml_version" option.
288 preserve
290 Since version 0.021
291 Default: false
293 This option is for loading and dumping.
295 Preserving scalar styles is still experimental.
297 use YAML::PP::Common qw/ PRESERVE_ORDER PRESERVE_SCALAR_STYLE /;
299 # Preserve the order of hash keys
301 my $yp = YAML::PP->new( preserve => PRESERVE_ORDER );
302 # Preserve the quoting style of scalars
304 my $yp = YAML::PP->new( preserve => PRESERVE_SCALAR_STYLE );
305 # Preserve block/flow style (since 0.024)
307 my $yp = YAML::PP->new( preserve => PRESERVE_FLOW_STYLE );
308 # Combine, e.g. preserve order and scalar style
310 my $yp = YAML::PP->new( preserve => PRESERVE_ORDER | PRESERVE_SCALAR_STYLE );
311 Do NOT rely on the internal implementation of it.
313 If you load the following input:
315 ---
317 z: 1
318 a: 2
319 ---
320 - plain
321 - 'single'
322 - "double"
323 - |
324 literal
325 ---
326 block mapping:
327 flow sequence: [a, b]
328 flow mapping: {a: b}
329 with this code:
331 my $yp = YAML::PP->new(
333 preserve => PRESERVE_ORDER | PRESERVE_SCALAR_STYLE | PRESERVE_FLOW_STYLE
334 );
335 my ($hash, $styles, $flow) = $yp->load_file($file);
336 $yp->dump_file($hash, $styles, $flow);
337 Then dumping it will return the same output. Only folded block
339 scalars '>' cannot preserve the style yet.
340 When loading, hashes will be tied to an internal class
342 ("YAML::PP::Preserve::Hash") that keeps the key order.
343 Scalars will be returned as objects of an internal class
345 ("YAML::PP::Preserve::Scalar") with overloading. If you assign to
346 such a scalar, the object will be replaced by a simple scalar.
347 # assignment, style gets lost
349 $styles->[1] .= ' append';
350 You can also pass 1 as a value. In this case all preserving options
352 will be enabled, also if there are new options added in the future.
353 There are also methods to craete preserved nodes from scratch. See
355 the "preserved_(scalar|mapping|sequence" "METHODS" below.
356 load_string
358 my $doc = $ypp->load_string("foo: bar");
359 my @docs = $ypp->load_string("foo: bar\n---\n- a");
360 Input should be Unicode characters.
362 So if you read from a file, you should decode it, for example with
364 "Encode::decode()".
365 Note that in scalar context, "load_string" and "load_file" return the
367 first document (like YAML::Syck), while YAML and YAML::XS return the
368 last.
369 load_file
371 my $doc = $ypp->load_file("file.yaml");
372 my @docs = $ypp->load_file("file.yaml");
373 Strings will be loaded as unicode characters.
375 dump_string
377 my $yaml = $ypp->dump_string($doc);
378 my $yaml = $ypp->dump_string($doc1, $doc2);
379 my $yaml = $ypp->dump_string(@docs);
380 Input strings should be Unicode characters.
382 Output will return Unicode characters.
384 So if you want to write that to a file (or pass to YAML::XS, for
386 example), you typically encode it via "Encode::encode()".
387 dump_file
389 $ypp->dump_file("file.yaml", $doc);
390 $ypp->dump_file("file.yaml", $doc1, $doc2);
391 $ypp->dump_file("file.yaml", @docs);
392 Input data should be Unicode characters.
394 dump
396 This will dump to a predefined writer. By default it will just use the
397 YAML::PP::Writer and output a string.
398 my $writer = MyWriter->new(\my $output);
400 my $yp = YAML::PP->new(
401 writer => $writer,
402 );
403 $yp->dump($data);
404 preserved_scalar
406 Since version 0.024
407 Experimental. Please report bugs or let me know this is useful and
409 works.
410 You can define a certain scalar style when dumping data. Figuring out
412 the best style is a hard task and practically impossible to get it
413 right for all cases. It's also a matter of taste.
414 use YAML::PP::Common qw/ PRESERVE_SCALAR_STYLE /;
416 my $yp = YAML::PP->new(
417 preserve => PRESERVE_SCALAR_STYLE,
418 );
419 # a single linebreak would normally be dumped with double quotes: "\n"
420 my $scalar = $yp->preserved_scalar("\n", style => YAML_LITERAL_SCALAR_STYLE );
421 my $data = { literal => $scalar };
423 my $dump = $yp->dump_string($data);
424 # output
425 ---
426 literal: |+
427 ...
429 preserved_mapping, preserved_sequence
431 Since version 0.024
432 Experimental. Please report bugs or let me know this is useful and
434 works.
435 With this you can define which nodes are dumped with the more compact
437 flow style instead of block style.
438 If you add "PRESERVE_ORDER" to the "preserve" option, it will also keep
440 the order of the keys in a hash.
441 use YAML::PP::Common qw/ PRESERVE_ORDER PRESERVE_FLOW_STYLE /;
443 my $yp = YAML::PP->new(
444 preserve => PRESERVE_FLOW_STYLE | PRESERVE_ORDER
445 );
446 my $hash = $yp->preserved_mapping({}, style => YAML_FLOW_MAPPING_STYLE);
448 # Add values after initialization to preserve order
449 %$hash = (z => 1, a => 2, y => 3, b => 4);
450 my $array = $yp->preserved_sequence([23, 24], style => YAML_FLOW_SEQUENCE_STYLE);
452 my $data = $yp->preserved_mapping({});
454 %$data = ( map => $hash, seq => $array );
455 my $dump = $yp->dump_string($data);
457 # output
458 ---
459 map: {z: 1, a: 2, y: 3, b: 4}
460 seq: [23, 24]
461 loader
463 Returns or sets the loader object, by default YAML::PP::Loader
464 dumper
466 Returns or sets the dumper object, by default YAML::PP::Dumper
467 schema
469 Returns or sets the schema object
470 default_schema
472 Creates and returns the default schema
473
475 The functions "Load", "LoadFile", "Dump" and "DumpFile" are provided as
476 a drop-in replacement for other existing YAML processors. No function
477 is exported by default.
478 Note that in scalar context, "Load" and "LoadFile" return the first
480 document (like YAML::Syck), while YAML and YAML::XS return the last.
481 Load
483 use YAML::PP qw/ Load /;
484 my $doc = Load($yaml);
485 my @docs = Load($yaml);
486 Works like "load_string".
488 LoadFile
490 use YAML::PP qw/ LoadFile /;
491 my $doc = LoadFile($file);
492 my @docs = LoadFile($file);
493 my @docs = LoadFile($filehandle);
494 Works like "load_file".
496 Dump
498 use YAML::PP qw/ Dump /;
499 my $yaml = Dump($doc);
500 my $yaml = Dump(@docs);
501 Works like "dump_string".
503 DumpFile
505 use YAML::PP qw/ DumpFile /;
506 DumpFile($file, $doc);
507 DumpFile($file, @docs);
508 DumpFile($filehandle, @docs);
509 Works like "dump_file".
511
513 You can alter the behaviour of YAML::PP by using the following schema
514 classes:
515 YAML::PP::Schema::Failsafe
517 One of the three YAML 1.2 official schemas
518 YAML::PP::Schema::JSON
520 One of the three YAML 1.2 official schemas.
521 YAML::PP::Schema::Core
523 One of the three YAML 1.2 official schemas. Default
524 YAML::PP::Schema::YAML1_1
526 Schema implementing the most common YAML 1.1 types
527 YAML::PP::Schema::Perl
529 Serializing Perl objects and types
530 YAML::PP::Schema::Binary
532 Serializing binary data
533 YAML::PP::Schema::Tie::IxHash
535 Deprecated. See option "preserve"
536 YAML::PP::Schema::Merge
538 YAML 1.1 merge keys for mappings
539 YAML::PP::Schema::Include
541 Include other YAML files via "!include" tags
542 To make the parsing process faster, you can plugin the libyaml parser
544 with YAML::PP::LibYAML.
545
547 The process of loading and dumping is split into the following steps:
548 Load:
550 YAML Stream Tokens Event List Data Structure
552 ---------> ---------> --------->
553 lex parse construct
554 Dump:
557 Data Structure Event List YAML Stream
559 ---------> --------->
560 represent emit
561 You can dump basic perl types like hashes, arrays, scalars (strings,
563 numbers). For dumping blessed objects and things like coderefs have a
564 look at YAML::PP::Perl/YAML::PP::Schema::Perl.
565 YAML::PP::Lexer
567 The Lexer is reading the YAML stream into tokens. This makes it
568 possible to generate syntax highlighted YAML output.
569 Note that the API to retrieve the tokens will change.
571 YAML::PP::Parser
573 The Parser retrieves the tokens from the Lexer. The main YAML
574 content is then parsed with the Grammar.
575 YAML::PP::Grammar
577 YAML::PP::Constructor
578 The Constructor creates a data structure from the Parser events.
579 YAML::PP::Loader
581 The Loader combines the constructor and parser.
582 YAML::PP::Dumper
584 The Dumper will delegate to the Representer
585 YAML::PP::Representer
587 The Representer will create Emitter events from the given data
588 structure.
589 YAML::PP::Emitter
591 The Emitter creates a YAML stream.
592 YAML::PP::Parser
594 Still TODO:
595 Implicit collection keys
597 ---
598 [ a, b, c ]: value
599 Implicit mapping in flow style sequences
601 ---
602 [ a, b, c: d ]
603 # equals
604 [ a, b, { c: d } ]
605 Plain mapping keys ending with colons
607 ---
608 key ends with two colons::: value
609 Supported Characters
611 If you have valid YAML that's not parsed, or the other way round,
612 please create an issue.
613 Line and Column Numbers
615 You will see line and column numbers in the error message. The
616 column numbers might still be wrong in some cases.
617 Error Messages
619 The error messages need to be improved.
620 Unicode Surrogate Pairs
622 Currently loaded as single characters without validating
623 Possibly more
625 YAML::PP::Constructor
627 The Constructor now supports all three YAML 1.2 Schemas, Failsafe, JSON
628 and Core. Additionally you can choose the schema for YAML 1.1 as
629 "YAML1_1".
630 Too see what strings are resolved as booleans, numbers, null etc. look
632 at <https://perlpunk.github.io/YAML-PP-p5/schema-examples.html>.
633 You can choose the Schema like this:
635 my $ypp = YAML::PP->new(schema => ['JSON']); # default is 'Core'
637 The Tags "!!seq" and "!!map" are still ignored for now.
639 It supports:
641 Handling of Anchors/Aliases
643 Like in modules like YAML, the Constructor will use references for
644 mappings and sequences, but obviously not for scalars.
645 YAML::XS uses real aliases, which allows also aliasing scalars. I
647 might add an option for that since aliasing is now available in
648 pure perl.
649 Boolean Handling
651 You can choose between 'perl' (1/'', currently default), 'JSON::PP'
652 and 'boolean'.pm for handling boolean types. That allows you to
653 dump the data structure with one of the JSON modules without losing
654 information about booleans.
655 Numbers
657 Numbers are created as real numbers instead of strings, so that
658 they are dumped correctly by modules like JSON::PP or JSON::XS, for
659 example.
660 Complex Keys
662 Mapping Keys in YAML can be more than just scalars. Of course, you
663 can't load that into a native perl structure. The Constructor will
664 stringify those keys with Data::Dumper instead of just returning
665 something like "HASH(0x55dc1b5d0178)".
666 Example:
668 use YAML::PP;
670 use JSON::PP;
671 my $ypp = YAML::PP->new;
672 my $coder = JSON::PP->new->ascii->pretty->allow_nonref->canonical;
673 my $yaml = <<'EOM';
674 complex:
675 ?
676 ?
677 a: 1
678 c: 2
679 : 23
680 : 42
681 EOM
682 my $data = $yppl->load_string($yaml);
683 say $coder->encode($data);
684 __END__
685 {
686 "complex" : {
687 "{'{a => 1,c => 2}' => 23}" : 42
688 }
689 }
690 TODO:
692 Parse Tree
694 I would like to generate a complete parse tree, that allows you to
695 manipulate the data structure and also dump it, including all
696 whitespaces and comments. The spec says that this is throwaway
697 content, but I read that many people wish to be able to keep the
698 comments.
699 YAML::PP::Dumper, YAML::PP::Emitter
701 The Dumper should be able to dump strings correctly, adding quotes
702 whenever a plain scalar would look like a special string, like "true",
703 or when it contains or starts with characters that are not allowed.
704 Most strings will be dumped as plain scalars without quotes. If they
706 contain special characters or have a special meaning, they will be
707 dumped with single quotes. If they contain control characters,
708 including <"\n">, they will be dumped with double quotes.
709 It will recognize JSON::PP::Boolean and boolean.pm objects and dump
711 them correctly.
712 Numbers which also have a PV flag will be recognized as numbers and not
714 as strings:
715 my $int = 23;
717 say "int: $int"; # $int will now also have a PV flag
718 That means that if you accidentally use a string in numeric context, it
720 will also be recognized as a number:
721 my $string = "23";
723 my $something = $string + 0;
724 print $yp->dump_string($string);
725 # will be emitted as an integer without quotes!
726 The layout is like libyaml output:
728 key:
730 - a
731 - b
732 - c
733 ---
734 - key1: 1
735 key2: 2
736 key3: 3
737 ---
738 - - a1
739 - a2
740 - - b1
741 - b2
742
744 All the available parsers and loaders for Perl are behaving
745 differently, and more important, aren't conforming to the spec.
746 YAML::XS is doing pretty well, but "libyaml" only handles YAML 1.1 and
747 diverges a bit from the spec. The pure perl loaders lack support for a
748 number of features.
749 I was going over YAML.pm issues end of 2016, integrating old patches
751 from rt.cpan.org and creating some pull requests myself. I realized
752 that it would be difficult to patch YAML.pm to parse YAML 1.1 or even
753 1.2, and it would also break existing usages relying on the current
754 behaviour.
755 In 2016 Ingy döt Net initiated two really cool projects:
757 "YAML TEST SUITE"
759 "YAML EDITOR"
760 These projects are a big help for any developer. So I got the idea to
762 write my own parser and started on New Year's Day 2017. Without the
763 test suite and the editor I would have never started this.
764 I also started another YAML Test project which allows one to get a
766 quick overview of which frameworks support which YAML features:
767 "YAML TEST MATRIX"
769 YAML TEST SUITE
771 <https://github.com/yaml/yaml-test-suite>
772 It contains about 230 test cases and expected parsing events and more.
774 There will be more tests coming. This test suite allows you to write
775 parsers without turning the examples from the Specification into tests
776 yourself. Also the examples aren't completely covering all cases - the
777 test suite aims to do that.
778 The suite contains .tml files, and in a separate 'data' release you
780 will find the content in separate files, if you can't or don't want to
781 use TestML.
782 Thanks also to Felix Krause, who is writing a YAML parser in Nim. He
784 turned all the spec examples into test cases.
785 YAML EDITOR
787 This is a tool to play around with several YAML parsers and loaders in
788 vim.
789 <https://github.com/yaml/yaml-editor>
791 The project contains the code to build the frameworks (16 as of this
793 writing) and put it into one big Docker image.
794 It also contains the yaml-editor itself, which will start a vim in the
796 docker container. It uses a lot of funky vimscript that makes playing
797 with it easy and useful. You can choose which frameworks you want to
798 test and see the output in a grid of vim windows.
799 Especially when writing a parser it is extremely helpful to have all
801 the test cases and be able to play around with your own examples to see
802 how they are handled.
803 YAML TEST MATRIX
805 I was curious to see how the different frameworks handle the test
806 cases, so, using the test suite and the docker image, I wrote some code
807 that runs the tests, manipulates the output to compare it with the
808 expected output, and created a matrix view.
809 <https://github.com/perlpunk/yaml-test-matrix>
811 You can find the latest build at <https://matrix.yaml.io>
813 As of this writing, the test matrix only contains valid test cases.
815 Invalid ones will be added.
816
818 Ingy döt Net
819 Ingy is one of the creators of YAML. In 2016 he started the YAML
820 Test Suite and the YAML Editor. He also made useful suggestions on
821 the class hierarchy of YAML::PP.
822 Felix "flyx" Krause
824 Felix answered countless questions about the YAML Specification.
825
827 YAML
828 YAML::XS
829 YAML::Syck
830 YAML::Tiny
831 YAML::PP::LibYAML
832 YAML::LibYAML::API
833 <https://www.yaml.info>
834
836 The Perl Foundation <https://www.perlfoundation.org/> sponsored this
837 project (and the YAML Test Suite) with a grant of 2500 USD in
838 2017-2018.
839
841 Copyright 2017-2020 by Tina Müller
842 This library is free software and may be distributed under the same
844 terms as perl itself.
845perl v5.32.0 2020-09-11 YAML::PP(3)