YAML::PP(3pm)

1YAML::PP(3)           User Contributed Perl Documentation          YAML::PP(3)
2
3
4

NAME

6       YAML::PP - YAML 1.2 processor
7

SYNOPSIS

9       WARNING: Most of the inner API is not stable yet.
10
11       Here are a few examples of the basic load and dump methods:
12
13           use YAML::PP;
14           my $ypp = YAML::PP->new;
15           my $yaml = <<'EOM';
16           --- # Document one is a mapping
17           name: Tina
18           age: 29
19           favourite language: Perl
20
21           --- # Document two is a sequence
22           - plain string
23           - 'in single quotes'
24           - "in double quotes we have escapes! like \t and \n"
25           - | # a literal block scalar
26             line1
27             line2
28           - > # a folded block scalar
29             this is all one
30             single line because the
31             linebreaks will be folded
32           EOM
33
34           my @documents = $ypp->load_string($yaml);
35           my @documents = $ypp->load_file($filename);
36
37           my $yaml = $ypp->dump_string($data1, $data2);
38           $ypp->dump_file($filename, $data1, $data2);
39
40           # The loader offers JSON::PP::Boolean, boolean.pm or
41           # perl 1/'' (currently default) for booleans
42           my $ypp = YAML::PP->new(boolean => 'JSON::PP');
43           my $ypp = YAML::PP->new(boolean => 'boolean');
44           my $ypp = YAML::PP->new(boolean => 'perl');
45
46           # Legacy interface
47           use YAML::PP qw/ Load Dump LoadFile DumpFile /;
48           my @documents = Load($yaml);
49           my @documents = LoadFile($filename);
50           my @documents = LoadFile($filehandle);
51           my $yaml = = Dump(@documents);
52           DumpFile($filename, @documents);
53           DumpFile($filenhandle @documents);
54
55           my $ypp = YAML::PP->new(schema => [qw/ + Perl /]);
56           my $yaml = $yp->dump_string($data_with_perl_objects);
57
58       Some utility scripts, mostly useful for debugging:
59
60           # Load YAML into a data structure and dump with Data::Dumper
61           yamlpp5-load < file.yaml
62
63           # Load and Dump
64           yamlpp5-load-dump < file.yaml
65
66           # Print the events from the parser in yaml-test-suite format
67           yamlpp5-events < file.yaml
68
69           # Parse and emit events directly without loading
70           yamlpp5-parse-emit < file.yaml
71
72           # Create ANSI colored YAML. Can also be useful for invalid YAML, showing
73           # you the exact location of the error
74           yamlpp5-highlight < file.yaml
75

DESCRIPTION

77       YAML::PP is a modular YAML processor.
78
79       It aims to support "YAML 1.2" and "YAML 1.1". See <http://yaml.org/>.
80       Some (rare) syntax elements are not yet supported and documented below.
81
82       YAML is a serialization language. The YAML input is called "YAML
83       Stream".  A stream consists of one or more "Documents", separated by a
84       line with a document start marker "---". A document optionally ends
85       with the document end marker "...".
86
87       This allows one to process continuous streams additionally to a fixed
88       input file or string.
89
90       The YAML::PP frontend will currently load all documents, and return
91       only the first if called with scalar context.
92
93       The YAML backend is implemented in a modular way that allows one to add
94       custom handling of YAML tags, perl objects and data types. The inner
95       API is not yet stable. Suggestions welcome.
96
97       You can check out all current parse and load results from the yaml-
98       test-suite here:
99       <https://perlpunk.github.io/YAML-PP-p5/test-suite.html>
100

PLUGINS

102       You can alter the behaviour of YAML::PP by using the following schema
103       classes:
104
105       YAML::PP::Schema::Failsafe
106           One of the three YAML 1.2 official schemas
107
108       YAML::PP::Schema::JSON
109           One of the three YAML 1.2 official schemas.
110
111       YAML::PP::Schema::Core
112           One of the three YAML 1.2 official schemas. Default
113
114       YAML::PP::Schema::YAML1_1
115           Schema implementing the most common YAML 1.1 types
116
117       YAML::PP::Schema::Perl
118           Serializing Perl objects and types
119
120       YAML::PP::Schema::Binary
121           Serializing binary data
122
123       YAML::PP::Schema::Tie::IxHash
124           Deprecated. See option "preserve"
125
126       YAML::PP::Schema::Merge
127           YAML 1.1 merge keys for mappings
128
129       YAML::PP::Schema::Include
130           Include other YAML files via "!include" tags
131
132       To make the parsing process faster, you can plugin the libyaml parser
133       with YAML::PP::LibYAML.
134

IMPLEMENTATION

136       The process of loading and dumping is split into the following steps:
137
138           Load:
139
140           YAML Stream        Tokens        Event List        Data Structure
141                     --------->    --------->        --------->
142                       lex           parse           construct
143
144
145           Dump:
146
147           Data Structure       Event List        YAML Stream
148                       --------->        --------->
149                       represent           emit
150
151       You can dump basic perl types like hashes, arrays, scalars (strings,
152       numbers).  For dumping blessed objects and things like coderefs have a
153       look at YAML::PP::Perl/YAML::PP::Schema::Perl.
154
155       YAML::PP::Lexer
156           The Lexer is reading the YAML stream into tokens. This makes it
157           possible to generate syntax highlighted YAML output.
158
159           Note that the API to retrieve the tokens will change.
160
161       YAML::PP::Parser
162           The Parser retrieves the tokens from the Lexer. The main YAML
163           content is then parsed with the Grammar.
164
165       YAML::PP::Grammar
166       YAML::PP::Constructor
167           The Constructor creates a data structure from the Parser events.
168
169       YAML::PP::Loader
170           The Loader combines the constructor and parser.
171
172       YAML::PP::Dumper
173           The Dumper will delegate to the Representer
174
175       YAML::PP::Representer
176           The Representer will create Emitter events from the given data
177           structure.
178
179       YAML::PP::Emitter
180           The Emitter creates a YAML stream.
181
182   YAML::PP::Parser
183       Still TODO:
184
185       Implicit collection keys
186               ---
187               [ a, b, c ]: value
188
189       Implicit mapping in flow style sequences
190               ---
191               [ a, b, c: d ]
192               # equals
193               [ a, b, { c: d } ]
194
195       Plain mapping keys ending with colons
196               ---
197               key ends with two colons::: value
198
199       Supported Characters
200           If you have valid YAML that's not parsed, or the other way round,
201           please create an issue.
202
203       Line and Column Numbers
204           You will see line and column numbers in the error message. The
205           column numbers might still be wrong in some cases.
206
207       Error Messages
208           The error messages need to be improved.
209
210       Unicode Surrogate Pairs
211           Currently loaded as single characters without validating
212
213       Possibly more
214
215   YAML::PP::Constructor
216       The Constructor now supports all three YAML 1.2 Schemas, Failsafe, JSON
217       and JSON.  Additionally you can choose the schema for YAML 1.1 as
218       "YAML1_1".
219
220       Too see what strings are resolved as booleans, numbers, null etc. look
221       at "t/31.schema.t".
222
223       You can choose the Schema, however, the API for that is not yet fixed.
224       Currently it looks like this:
225
226           my $ypp = YAML::PP->new(schema => ['JSON']); # default is 'Core'
227
228       The Tags "!!seq" and "!!map" are still ignored for now.
229
230       It supports:
231
232       Handling of Anchors/Aliases
233           Like in modules like YAML, the Constructor will use references for
234           mappings and sequences, but obviously not for scalars.
235
236       Boolean Handling
237           You can choose between 'perl' (1/'', currently default), 'JSON::PP'
238           and 'boolean'.pm for handling boolean types.  That allows you to
239           dump the data structure with one of the JSON modules without losing
240           information about booleans.
241
242       Numbers
243           Numbers are created as real numbers instead of strings, so that
244           they are dumped correctly by modules like JSON::PP or JSON::XS, for
245           example.
246
247           See "NUMBERS" for an example.
248
249       Complex Keys
250           Mapping Keys in YAML can be more than just scalars. Of course, you
251           can't load that into a native perl structure. The Constructor will
252           stringify those keys with Data::Dumper instead of just returning
253           something like "HASH(0x55dc1b5d0178)".
254
255           Example:
256
257               use YAML::PP;
258               use JSON::PP;
259               my $ypp = YAML::PP->new;
260               my $coder = JSON::PP->new->ascii->pretty->allow_nonref->canonical;
261               my $yaml = <<'EOM';
262               complex:
263                   ?
264                       ?
265                           a: 1
266                           c: 2
267                       : 23
268                   : 42
269               EOM
270               my $data = $yppl->load_string($yaml);
271               say $coder->encode($data);
272               __END__
273               {
274                  "complex" : {
275                     "{'{a => 1,c => 2}' => 23}" : 42
276                  }
277               }
278
279       TODO:
280
281       Parse Tree
282           I would like to generate a complete parse tree, that allows you to
283           manipulate the data structure and also dump it, including all
284           whitespaces and comments.  The spec says that this is throwaway
285           content, but I read that many people wish to be able to keep the
286           comments.
287
288   YAML::PP::Dumper, YAML::PP::Emitter
289       The Dumper should be able to dump strings correctly, adding quotes
290       whenever a plain scalar would look like a special string, like "true",
291       or when it contains or starts with characters that are not allowed.
292
293       Most strings will be dumped as plain scalars without quotes. If they
294       contain special characters or have a special meaning, they will be
295       dumped with single quotes. If they contain control characters,
296       including <"\n">, they will be dumped with double quotes.
297
298       It will recognize JSON::PP::Boolean and boolean.pm objects and dump
299       them correctly.
300
301       Numbers which also have a PV flag will be recognized as numbers and not
302       as strings:
303
304           my $int = 23;
305           say "int: $int"; # $int will now also have a PV flag
306
307       That means that if you accidentally use a string in numeric context, it
308       will also be recognized as a number:
309
310           my $string = "23";
311           my $something = $string + 0;
312           print $yp->dump_string($string);
313           # will be emitted as an integer without quotes!
314
315       The layout is like libyaml output:
316
317           key:
318           - a
319           - b
320           - c
321           ---
322           - key1: 1
323             key2: 2
324             key3: 3
325           ---
326           - - a1
327             - a2
328           - - b1
329             - b2
330

METHODS

332       new
333               my $ypp = YAML::PP->new;
334               # load booleans via boolean.pm
335               my $ypp = YAML::PP->new( boolean => 'boolean' );
336               # load booleans via JSON::PP::true/false
337               my $ypp = YAML::PP->new( boolean => 'JSON::PP' );
338
339               # use YAML 1.2 Failsafe Schema
340               my $ypp = YAML::PP->new( schema => ['Failsafe'] );
341               # use YAML 1.2 JSON Schema
342               my $ypp = YAML::PP->new( schema => ['JSON'] );
343               # use YAML 1.2 Core Schema
344               my $ypp = YAML::PP->new( schema => ['Core'] );
345
346               # Die when detecting cyclic references
347               my $ypp = YAML::PP->new( cyclic_refs => 'fatal' );
348
349               my $ypp = YAML::PP->new(
350                   boolean => 'JSON::PP',
351                   schema => ['Core'],
352                   cyclic_refs => 'fatal',
353                   indent => 4,
354                   header => 1,
355                   footer => 1,
356                   version_directive => 1,
357               );
358
359           Options:
360
361           boolean
362               Values: "perl" (currently default), "JSON::PP", "boolean"
363
364           schema
365               Default: "['Core']"
366
367               Array reference. Here you can define what schema to use.
368               Supported standard Schemas are: "Failsafe", "JSON", "Core",
369               "YAML1_1".
370
371               To get an overview how the different Schemas behave, see
372               <https://perlpunk.github.io/YAML-PP-p5/schemas.html>
373
374               Additionally you can add further schemas, for example "Merge".
375
376           cyclic_refs
377               Default: 'allow' but will be switched to fatal in the future
378               for safety!
379
380               Defines what to do when a cyclic reference is detected when
381               loading.
382
383                   # fatal  - die
384                   # warn   - Just warn about them and replace with undef
385                   # ignore - replace with undef
386                   # allow  - Default
387
388           indent
389               Default: 2
390
391               Use that many spaces for indenting
392
393           header
394               Default: 1
395
396               Print document heaader "---"
397
398           footer
399               Default: 0
400
401               Print document footer "..."
402
403           yaml_version
404               Default: 1.2
405
406               Note that in this case, a directive "%YAML 1.1" will basically
407               be ignored and everything loaded with the "1.2 Core" Schema.
408
409               If you want to support both YAML 1.1 and 1.2, you have to
410               specify that, and the schema ("Core" or "YAML1_1") will be
411               chosen automatically.
412
413                   my $yp = YAML::PP->new(
414                       yaml_version => ['1.2', '1.1'],
415                   );
416
417               This is the same as
418
419                   my $yp = YAML::PP->new(
420                       schema => ['+'],
421                       yaml_version => ['1.2', '1.1'],
422                   );
423
424               because the "+" stands for the default schema per version.
425
426               When loading, and there is no %YAML directive, 1.2 will be
427               considered as default, and the "Core" schema will be used.
428
429               If there is a "%YAML 1.1" directive, the "YAML1_1" schema will
430               be used.
431
432               Of course, you can also make 1.1 the default:
433
434                   my $yp = YAML::PP->new(
435                       yaml_version => ['1.1', '1.2'],
436                   );
437
438               You can also specify 1.1 only:
439
440                   my $yp = YAML::PP->new(
441                       yaml_version => ['1.1'],
442                   );
443
444               In this case also documents with "%YAML 1.2" will be loaded
445               with the "YAML1_1" schema.
446
447           version_directive
448               Default: 0
449
450               Print Version Directive "%YAML 1.2" (or "%YAML 1.1") on top of
451               each YAML document. It will use the first version specified in
452               the "yaml_version" option.
453
454           preserve (since 0.021)
455               Experimental. Default: false
456
457                   use YAML::PP::Common qw/ PRESERVE_ORDER PRESERVE_SCALAR_STYLE /;
458                   # Preserve the order of hash keys
459                   my $yp = YAML::PP->new( preserve => PRESERVE_ORDER );
460                   # Preserve the quoting style of scalars
461                   my $yp = YAML::PP->new( preserve => PRESERVE_SCALAR_STYLE );
462                   # Preserve order and scalar style
463                   my $yp = YAML::PP->new( preserve => PRESERVE_ORDER | PRESERVE_SCALAR_STYLE );
464
465               Do NOT rely on the internal implementation of it.
466
467               If you load the following input:
468
469                   ---
470                   z: 1
471                   a: 2
472                   ---
473                   - plain
474                   - 'single'
475                   - "double"
476                   - |
477                     literal
478
479                   my $yp = YAML::PP->new( preserve => PRESERVE_ORDER | PRESERVE_SCALAR_STYLE );
480                   my ($hash, $styles) = $yp->load_file($file);
481
482               Then dumping it will return the same output.  Only folded block
483               scalars '>' cannot preserve the style yet.
484
485               When loading, hashes will be tied to an internal class
486               ("YAML::PP::Preserve::Hash") that keeps the key order.
487
488               Scalars will be returned as objects of an internal class
489               ("YAML::PP::Preserve::Scalar") with overloading. If you assign
490               to such a scalar, the object will be replaced by a simple
491               scalar.
492
493                   # assignment, style gets lost
494                   $styles->[1] .= ' append';
495
496       load_string
497               my $doc = $ypp->load_string("foo: bar");
498               my @docs = $ypp->load_string("foo: bar\n---\n- a");
499
500           Input should be Unicode characters.
501
502           So if you read from a file, you should decode it, for example with
503           "Encode::decode()".
504
505           Note that in scalar context, "load_string" and "load_file" return
506           the first document (like YAML::Syck), while YAML and YAML::XS
507           return the last.
508
509       load_file
510               my $doc = $ypp->load_file("file.yaml");
511               my @docs = $ypp->load_file("file.yaml");
512
513           Strings will be loaded as unicode characters.
514
515       dump_string
516               my $yaml = $ypp->dump_string($doc);
517               my $yaml = $ypp->dump_string($doc1, $doc2);
518               my $yaml = $ypp->dump_string(@docs);
519
520           Input strings should be Unicode characters.
521
522           Output will return Unicode characters.
523
524           So if you want to write that to a file (or pass to YAML::XS, for
525           example), you typically encode it via "Encode::encode()".
526
527       dump_file
528               $ypp->dump_file("file.yaml", $doc);
529               $ypp->dump_file("file.yaml", $doc1, $doc2);
530               $ypp->dump_file("file.yaml", @docs);
531
532           Input data should be Unicode characters.
533
534       dump
535           This will dump to a predefined writer. By default it will just use
536           the YAML::PP::Writer and output a string.
537
538               my $writer = MyWriter->new(\my $output);
539               my $yp = YAML::PP->new(
540                   writer => $writer,
541               );
542               $yp->dump($data);
543
544       loader
545           Returns or sets the loader object, by default YAML::PP::Loader
546
547       dumper
548           Returns or sets the dumper object, by default YAML::PP::Dumper
549
550       schema
551           Returns or sets the schema object
552
553       default_schema
554           Creates and returns the default schema
555

FUNCTIONS

557       The functions "Load", "LoadFile", "Dump" and "DumpFile" are provided as
558       a drop-in replacement for other existing YAML processors.  No function
559       is exported by default.
560
561       Note that in scalar context, "Load" and "LoadFile" return the first
562       document (like YAML::Syck), while YAML and YAML::XS return the last.
563
564       Load
565               use YAML::PP qw/ Load /;
566               my $doc = Load($yaml);
567               my @docs = Load($yaml);
568
569           Works like "load_string".
570
571       LoadFile
572               use YAML::PP qw/ LoadFile /;
573               my $doc = LoadFile($file);
574               my @docs = LoadFile($file);
575               my @docs = LoadFile($filehandle);
576
577           Works like "load_file".
578
579       Dump
580               use YAML::PP qw/ Dump /;
581               my $yaml = Dump($doc);
582               my $yaml = Dump(@docs);
583
584           Works like "dump_string".
585
586       DumpFile
587               use YAML::PP qw/ DumpFile /;
588               DumpFile($file, $doc);
589               DumpFile($file, @docs);
590               DumpFile($filehandle, @docs);
591
592           Works like "dump_file".
593

NUMBERS

595       Compare the output of the following YAML Loaders and JSON::PP dump:
596
597           use JSON::PP;
598           use Devel::Peek;
599
600           use YAML::XS ();
601           use YAML ();
602               $YAML::Numify = 1; # since version 1.23
603           use YAML::Syck ();
604               $YAML::Syck::ImplicitTyping = 1;
605           use YAML::Tiny ();
606           use YAML::PP;
607
608           my $yaml = "foo: 23";
609
610           my $d1 = YAML::XS::Load($yaml);
611           my $d2 = YAML::Load($yaml);
612           my $d3 = YAML::Syck::Load($yaml);
613           my $d4 = YAML::Tiny->read_string($yaml)->[0];
614           my $d5 = YAML::PP->new->load_string($yaml);
615
616           Dump $d1->{foo};
617           Dump $d2->{foo};
618           Dump $d3->{foo};
619           Dump $d4->{foo};
620           Dump $d5->{foo};
621
622           say encode_json($d1);
623           say encode_json($d2);
624           say encode_json($d3);
625           say encode_json($d4);
626           say encode_json($d5);
627
628           SV = PVIV(0x55bbaff2bae0) at 0x55bbaff26518
629             REFCNT = 1
630             FLAGS = (IOK,POK,pIOK,pPOK)
631             IV = 23
632             PV = 0x55bbb06e67a0 "23"\0
633             CUR = 2
634             LEN = 10
635           SV = PVMG(0x55bbb08959b0) at 0x55bbb08fc6e8
636             REFCNT = 1
637             FLAGS = (IOK,pIOK)
638             IV = 23
639             NV = 0
640             PV = 0
641           SV = IV(0x55bbaffcb3b0) at 0x55bbaffcb3c0
642             REFCNT = 1
643             FLAGS = (IOK,pIOK)
644             IV = 23
645           SV = PVMG(0x55bbaff2f1f0) at 0x55bbb08fc8c8
646             REFCNT = 1
647             FLAGS = (POK,pPOK,UTF8)
648             IV = 0
649             NV = 0
650             PV = 0x55bbb0909d00 "23"\0 [UTF8 "23"]
651             CUR = 2
652             LEN = 10
653           SV = PVMG(0x55bbaff2f6d0) at 0x55bbb08b2c10
654             REFCNT = 1
655             FLAGS = (IOK,pIOK)
656             IV = 23
657             NV = 0
658             PV = 0
659
660           {"foo":"23"}
661           {"foo":23}
662           {"foo":23}
663           {"foo":"23"}
664           {"foo":23}
665

WHY

667       All the available parsers and loaders for Perl are behaving
668       differently, and more important, aren't conforming to the spec.
669       YAML::XS is doing pretty well, but "libyaml" only handles YAML 1.1 and
670       diverges a bit from the spec. The pure perl loaders lack support for a
671       number of features.
672
673       I was going over YAML.pm issues end of 216, integrating old patches
674       from rt.cpan.org and creating some pull requests myself. I realized
675       that it would be difficult to patch YAML.pm to parse YAML 1.1 or even
676       1.2, and it would also break existing usages relying on the current
677       behaviour.
678
679       In 2016 Ingy döt Net initiated two really cool projects:
680
681       "YAML TEST SUITE"
682       "YAML EDITOR"
683
684       These projects are a big help for any developer. So I got the idea to
685       write my own parser and started on New Year's Day 2017.  Without the
686       test suite and the editor I would have never started this.
687
688       I also started another YAML Test project which allows one to get a
689       quick overview of which frameworks support which YAML features:
690
691       "YAML TEST MATRIX"
692
693   YAML TEST SUITE
694       <https://github.com/yaml/yaml-test-suite>
695
696       It contains about 230 test cases and expected parsing events and more.
697       There will be more tests coming. This test suite allows you to write
698       parsers without turning the examples from the Specification into tests
699       yourself.  Also the examples aren't completely covering all cases - the
700       test suite aims to do that.
701
702       The suite contains .tml files, and in a separate 'data' branch you will
703       find the content in separate files, if you can't or don't want to use
704       TestML.
705
706       Thanks also to Felix Krause, who is writing a YAML parser in Nim.  He
707       turned all the spec examples into test cases.
708
709   YAML EDITOR
710       This is a tool to play around with several YAML parsers and loaders in
711       vim.
712
713       <https://github.com/yaml/yaml-editor>
714
715       The project contains the code to build the frameworks (16 as of this
716       writing) and put it into one big Docker image.
717
718       It also contains the yaml-editor itself, which will start a vim in the
719       docker container. It uses a lot of funky vimscript that makes playing
720       with it easy and useful. You can choose which frameworks you want to
721       test and see the output in a grid of vim windows.
722
723       Especially when writing a parser it is extremely helpful to have all
724       the test cases and be able to play around with your own examples to see
725       how they are handled.
726
727   YAML TEST MATRIX
728       I was curious to see how the different frameworks handle the test
729       cases, so, using the test suite and the docker image, I wrote some code
730       that runs the tests, manipulates the output to compare it with the
731       expected output, and created a matrix view.
732
733       <https://github.com/perlpunk/yaml-test-matrix>
734
735       You can find the latest build at <http://matrix.yaml.io>
736
737       As of this writing, the test matrix only contains valid test cases.
738       Invalid ones will be added.
739

CONTRIBUTORS

741       Ingy döt Net
742           Ingy is one of the creators of YAML. In 2016 he started the YAML
743           Test Suite and the YAML Editor. He also made useful suggestions on
744           the class hierarchy of YAML::PP.
745
746       Felix "flyx" Krause
747           Felix answered countless questions about the YAML Specification.
748

COPYRIGHT AND LICENSE

761       Copyright 2018 by Tina Müller
762
763       This library is free software and may be distributed under the same
764       terms as perl itself.
765
766
767
768perl v5.30.1                      2020-03-02                       YAML::PP(3)

NAME

SYNOPSIS

DESCRIPTION

PLUGINS

IMPLEMENTATION

METHODS

FUNCTIONS

NUMBERS

WHY

CONTRIBUTORS

SEE ALSO

SPONSORS

COPYRIGHT AND LICENSE