1Pegex::Miscellany(3)  User Contributed Perl Documentation Pegex::Miscellany(3)
2
3
4

Miscellany

6       This document contains things about Pegex that were written but seemed
7       out of place in their original documents. Still they are possibly
8       useful so live here for now.
9

Pegex Overview

11       In the diagram below, there is a simple language called Foo. The
12       diagram shows how Pegex can take a text grammar defining Foo and
13       generate a parser that can parse Foo sources into data (abstract syntax
14       trees).
15
16                                     Parsing a language called "Foo"
17                                        with the Pegex toolset.
18
19                                       .-----------------------.
20           .--------------------.      |    Pegex::Compiler    |
21           |    Foo Language    |      |-----------------------|    Serialize
22           |--------------------|----->| Pegex::Grammar::Pegex |---------.
23           | Pegex grammar text |      | Pegex::Receiver       |         |
24           '--------------------'      '-----------------------'         v
25           ......................                  |                 .------.
26           |                    |                  | compile()       | YAML |
27           |foo: verb noun      |                  v                 '------'
28           |verb: /Hello/       |       .--------------------.       .------.
29           |noun: /world/       |       | Foo grammar tree   |       | JSON |
30           |                    |       '--------------------'       '------'
31           ......................                  |                 .------.
32                                                   |                 | Perl |
33                                                   v                 '------'
34                                        .---------------------.      .--------.
35                                        | Pegex::Grammar::Foo |      | Python |
36                                        |---------------------|      '--------'
37                                        | Pegex::Parser       |      .-----.
38                                        | Pegex::AST::Foo     |      | etc |
39            .-----------------.         '---------------------'      '-----'
40            |  Foo Language   |                    |
41            |-----------------|------------------->| parse()
42            | Foo source text |                    v
43            '-----------------'        .----------------------.
44            ...................        | Parsed Foo Data Tree |
45            |Hello world      |        '----------------------'
46            ...................        ........................
47                                       |- verb: Hello         |
48                                       |- noun: world         |
49                                       ........................
50

FYI

52       Pegex is self-hosting. This means that the Pegex grammar language
53       syntax is defined by a Pegex grammar! This is important because (just
54       like any Pegex based language) it makes it easier to port to new
55       programming languages. You can find the Pegex grammar for Pegex
56       grammars here: <http://github.com/ingydotnet/pegex-pgx/>.
57
58       Pegex was originally inspired by Perl 6 Rules. It also takes ideas from
59       Damian Conway's Perl 5 module, Regexp::Grammars. Pegex tries to take
60       the best ideas from these great works, and make them work in as many
61       languages as possible. That's Acmeism.
62

Self Compilation Tricks

64       You can have some fun using Pegex to compile itself. First get the
65       Pegex grammar repo:
66
67           git clone git://github.com/ingydotnet/pegex-pgx.git
68           cd pegex-pgx
69
70       Then parse and dump the Pegex grammar with Pegex:
71
72           perl -MXXX -MPegex -e 'XXX pegex("pegex.pgx")->parse("pegex.pgx")'
73
74       For a different view of the data tree, try:
75
76           perl -MXXX -MPegex -e 'XXX pegex("pegex.pgx", receiver => "Pegex::Tree")->parse("pegex.pgx")'
77
78       Finally to emulate the Pegex compiler do this:
79
80           perl -MXXX -MPegex -e 'XXX pegex("pegex.pgx", receiver => "Pegex::Pegex::AST")->parse("pegex.pgx")'
81
82       This specifies a "receiving" class that can shape the results into
83       something useful. Indeed, this is the exact guts of
84       Pegex::Grammar::Pegex.
85

A Real World EXAMPLE

87       TestML is a new Acmeist unit test language. It is perfect for software
88       that needs to run equivalently in more than one language. In fact,
89       Pegex itself is tested with TestML!!
90
91       TestML has a language specification grammar:
92       <http://www.testml.org/specification/language/>
93
94       The Perl6 implementation of TestML uses this grammar in:
95       <https://github.com/ingydotnet/testml-pm6/blob/master/lib/TestML/Parser/Grammar.pm>
96
97       All other implementations of TestML use this Pegex grammar:
98       <https://github.com/ingydotnet/testml-pgx/blob/master/testml.pgx>
99
100       In Perl 5, Pegex::Compiler is used to compile the grammar into this
101       simple data structure (shown in YAML):
102       <https://github.com/ingydotnet/testml-pgx/blob/master/testml.pgx.yaml>
103
104       The grammar can also be precompiled to JSON:
105       <https://github.com/ingydotnet/testml-pgx/blob/master/testml.pgx.json>
106
107       Pegex::Compiler further compiles this into a Perl 5 only grammar tree,
108       which becomes this module:
109       <https://github.com/ingydotnet/testml-pm/blob/master/lib/TestML/Compiler/Pegex/Grammar.pm>
110
111       TestML::Parser::Grammar is a subclass of Pegex::Grammar. It can be used
112       to parse TestML files. TestML::Parser calls the parse() method of the
113       grammar with a TestML::AST object that receives callbacks when various
114       rules match, and uses the information to build a TestML::Document
115       object.
116
117       Thus TestML is an Acmeist language written in Pegex. It can be easily
118       ported to every language where Pegex exists. In fact, it must be ported
119       to those languages in order to test the new Pegex implementation!
120
121
122
123perl v5.32.1                      2021-01-27              Pegex::Miscellany(3)
Impressum