1XS::Parse::Infix(3) User Contributed Perl Documentation XS::Parse::Infix(3)
2
6 "XS::Parse::Infix" - XS functions to assist in parsing infix operators
7
9 This module provides some XS functions to assist in writing syntax
10 modules that provide new infix operators as perl syntax, primarily for
11 authors of syntax plugins. It is unlikely to be of much use to anyone
12 else; and highly unlikely to be of any use when writing perl code using
13 these. Unless you are writing a syntax plugin using XS, this module is
14 not for you.
15 This module is also currently experimental, and the design is still
17 evolving and subject to change. Later versions may break ABI
18 compatibility, requiring changes or at least a rebuild of any module
19 that depends on it.
20 In addition, the places this functionality can be used are relatively
22 small. No current release of perl actually supports custom infix
23 operators, though it has now been merged to "blead", the main
24 development branch, and will be present in development release
25 "v5.37.7" onwards.
26 In addition, the various "XPK_INFIX_*" token types of
28 XS::Parse::Keyword support querying on this module, so some syntax
29 provided by other modules may be able to make use of these new infix
30 operators.
31
33 HAVE_PL_INFIX_PLUGIN
34 if( XS::Parse::Infix::HAVE_PL_INFIX_PLUGIN ) { ... }
35 This constant is true if built on a perl that supports the
37 "PL_infix_plugin" extension mechanism, meaning that custom infix
38 operators registered with this module will actually be recognised by
39 the perl parser.
40 No actual production releases of perl yet support this feature, but see
42 above for details of development versions which do.
43
45 boot_xs_parse_infix
46 void boot_xs_parse_infix(double ver);
47 Call this function from your "BOOT" section in order to initialise the
49 module and parsing hooks.
50 ver should either be 0 or a decimal number for the module version
52 requirement; e.g.
53 boot_xs_parse_infix(0.14);
55 parse_infix
57 bool parse_infix(enum XSParseInfixSelection select, struct XSParseInfixInfo **infop);
58 Since version 0.27.
60 This function attempts to parse syntax for an infix operator from the
62 current parser position. If it is successful, it fills in the variable
63 pointed to by infop with a pointer to the actual information structure
64 and returns "true". If no suitable operator is found, returns "false".
65 xs_parse_infix_new_op
67 OP *xs_parse_infix_new_op(const struct XSParseInfixInfo *info, U32 flags,
68 OP *lhs, OP *rhs);
69 This function constructs a new optree fragment to represent invoking
71 the infix operator with the given operands. It should be used much the
72 same as core perl's "newBINOP" function.
73 The "info" structure pointer would be obtained from the "infix" field
75 of the result of invoking the various "XPK_INFIX_*" token types from
76 "XS::Parse::Keyword", or by calling "parse_infix" directly.
77 register_xs_parse_infix
79 void register_xs_parse_infix(const char *opname,
80 const struct XSParseInfixHooks *hooks, void *hookdata);
81 This function installs a set of parsing hooks to be associated with the
83 given operator name. This new operator will then be available via
84 XS::Parse::Keyword by the various "XPK_INFIX_*" token types,
85 "parse_infix", or to core perl's "PL_infix_plugin" if available.
86 These tokens will all yield an info structure, with the following
88 fields:
89 struct XSParseInfixInfo {
91 const char *opname;
92 OPCODE opcode; /* for built-in operators, or OP_CUSTOM for
93 custom-registered ones */
94 struct XSParseInfixHooks *hooks;
96 void *hookdata;
97 enum XSParseInfixClassification cls; /* since version 0.28 */
99 };
100 If the operator name contains any non-ASCII characters they are
102 presumed to be in UTF-8 encoding. This will matter for deparse
103 purposes.
104
106 The "XSParseInfixHooks" structure provides the following fields which
107 are used at various stages of parsing.
108 struct XSParseInfixHooks {
110 U16 flags; /* currently ignored */
111 U8 lhs_flags;
112 U8 rhs_flags;
113 enum XSParseInfixClassification cls;
114 const char *wrapper_func_name;
116 const char *permit_hintkey;
118 bool (*permit)(pTHX_ void *hookdata);
119 OP *(*new_op)(pTHX_ U32 flags, OP *lhs, OP *rhs, SV **parsedata, void *hookdata);
121 OP *(*ppaddr)(pTHX);
122 /* optional */
124 void (*parse)(pTHX_ U32 flags, SV **parsedata, void *hookdata);
125 };
126 Flags
128 The "flags" field is currently ignored. It is defined simply to reserve
129 the space in case used in a later version. It should be set to zero.
130 The "lhs_flags" and "rhs_flags" fields give details on how to handle
132 the left- and right-hand side operands, respectively.
133 It should be set to one of the following constants, or left as zero:
135 XPI_OPERAND_TERM_LIST
137 The operand will be foced into list context, preserving the
138 "OP_PUSHMARK" at the beginning. This means that the ppfunc for this
139 infix operator will have to "POPMARK" to find that.
140 XPI_OPERAND_LIST
142 The same as above.
143 Older versions used to provide constants named "XPI_OPERAND_ARITH" and
145 "XPI_OPERAND_TERM" but they related to an older version of the core
146 perl branch. These names are now aliases for zero, and can be removed
147 from new code.
148 In addition the following extra bitflags are defined:
150 XPI_OPERAND_ONLY_LOOK
152 If set, the operator function promises that it will not mutate any
153 of its passed values, nor allow leaking of direct alias pointers to
154 them via return value or other locations.
155 This flag is optional; omitting it when applicable will not change
157 any observed behaviour. Setting it may enable certain optimisations
158 to be performed.
159 Currently, this flag simply enables an optimisation in the call-
161 checker for infix operator wrapper functions that take list-shaped
162 operands. This optimisation discards an "OP_ANONLIST" operation
163 which would create a temporary anonymous array reference for its
164 operand values, allowing a slight saving of memory use and CPU
165 time. This optimisation is only safe to perform if the operator
166 does not mutate or retain aliases of any of the arguments, as
167 otherwise the caller might see unexpected modifications or value
168 references to the values passed.
169 The Selection Stage
171 The "cls" field gives a "classification" of the operator, suggesting
172 what sort of operation it provides. This is used as a filter by the
173 various "XS::Parse::Keyword" selection macros.
174 The classification should be one of the "XPI_CLS_*" constants found and
176 described further in the main XSParseInfix.h file.
177 The "permit" Stage
179 As a shortcut for the common case, the "permit_hintkey" may point to a
180 string to look up from the hints hash. If the given key name is not
181 found in the hints hash then the keyword is not permitted. If the key
182 is present then the "permit" function is invoked as normal.
183 If not rejected by a hint key that was not found in the hints hash, the
185 function part of the stage is called next and should inspect whether
186 the keyword is permitted at this time perhaps by inspecting other
187 lexical clues, and return true only if the keyword is permitted.
188 Both the string and the function are optional. Either or both may be
190 present. If neither is present then the keyword is always permitted -
191 which is likely not what you wanted to do.
192 The "parse" Stage
194 If the optional "parse" hook function is present, it is called
195 immediately after the parser has recognised the presence of the named
196 operator itself but before it attempts to consume the right-hand side
197 term. This hook function can attempt further parsing, in order to
198 implement more complex syntax such as hyper-operators.
199 When invoked, it is passed a pointer to an "SV *"-typed storage
201 variable. It is free to use this variable it desires to store a result,
202 which will then later be made available to the "new_op" function.
203 The Op Generation Stage
205 If the infix operator is going to be used, then one of the "new_op" or
206 the "ppaddr" fields explain how to create a new optree fragment.
207 If "new_op" is defined then it will be used, and is expected to return
209 an optree fragment that consumes the LHS and RHS arguments to implement
210 the semantics of the operator. If the optional "parse" stage had been
211 present earlier, the "SV **" pointer passed here will point to the same
212 storage that "parse" had previously had access to, so it can retrieve
213 the results.
214 If "new_op" is not present, then the "ppaddr" will be used instead to
216 construct a new BINOP of the "OP_CUSTOM" type. If an earlier "parse"
217 stage had stored additional results into the "SV *" variable these will
218 be lost here.
219 The Wrapper Function
221 Additionally, if the "wrapper_func_name" field is set to a string, this
222 gives the (fully-qualified) name for a function to be generated as part
223 of registering the operator. This newly-generated function will act as
224 a wrapper for the operator.
225 For operators whose RHS is a scalar, the wrapper function is assumed to
227 take two simple scalar arguments. The result of invoking the function
228 on those arguments will be determined by using the operator code.
229 $result = $lhs OP $rhs;
231 $result = WRAPPERFUNC( $lhs, $rhs );
233 For operators whose RHS is a list, the wrapper function takes at least
235 one argument, possibly more. The first argument is the scalar on the
236 LHS, and the remaining arguments, however many there are, form the RHS:
237 $result = $lhs OP @rhs;
239 $result = WRAPPERFUNC( $lhs, @rhs );
241 For operators whose LHS and RHS is a list, the wrapper function takes
243 two arguments which must be array references containing the lists.
244 $result = @lhs OP @rhs;
246 $result = WRAPPERFUNC( \@lhs, \@rhs );
248 This creates a convenience for accessing the operator from perls that
250 do not support "PL_infix_plugin".
251 In the case of scalar infix operators, the wrapper function also
253 includes a call-checker which attempts to inline the operator directly
254 into the callsite. Thus, in simple cases where the function is called
255 directly on exactly two scalar arguments (such as in the following), no
256 "ENTERSUB" overhead will be incurred and the generated optree will be
257 identical to that which would have been generated by using infix
258 operator syntax directly:
259 WRAPPERFUNC( $lhs, $rhs );
261 WRAPPERFUNC( $lhs, CONSTANT );
262 WRAPPERFUNC( $args[0], $args[1] );
263 WRAPPERFUNC( $lhs, scalar otherfunc() );
264 The checker is very pessimistic and will only rewrite callsites where
266 it determines this can be done safely. It will not rewrite any of the
267 following forms:
268 WRAPPERFUNC( $onearg ); # not enough args
270 WRAPPERFUNC( $x, $y, $z ); # too many args
271 WRAPPERFUNC( @args[0,1] ); # not a scalar
272 WRAPPERFUNC( $lhs, otherfunc() ); # not a scalar
273 The wrapper function for infix operators which take lists on both sides
275 also has a call-checker which will attempt to inline the operator in
276 similar circumstances. In addition to the optimisations described above
277 for scalar operators, this checker will also inline an array-reference
278 operator and omit the resulting dereference behaviour. Thus, the two
279 following lines emit the same optree, without an "OP_SREFGEN" or
280 "OP_RV2AV":
281 @lhs OP @rhs;
283 WRAPPERFUNC( \@lhs, \@rhs );
284 Note that technically, this optimisation isn't strictly transparent in
286 the odd cornercase that one of the referenced arrays is also the
287 backing store for a blessed object reference, and that object class has
288 a "@{}" overload.
289 my @arr;
291 package SomeClass {
292 use overload '@{}' => sub { return ["values", "go", "here"]; };
293 }
294 bless \@arr, "SomeClass";
295 # this will not actually invoke the overload operator
297 WRAPPERFUNC( \@arr, [4, 5, 6] );
298 As this cornercase relates to taking duplicate references to the same
300 blessed object's backing store variable, it should not matter to any
301 real code; regular objects that are passed by reference into the
302 wrapper function will run their overload methods as normal.
303 The callchecker for list operands can optionally also discard an op of
305 the "OP_ANONLIST" type, which is used by anonymous array-ref
306 construction:
307 ($u, $v, $w) OP ($x, $y, $z);
309 WRAPPERFUNC( [$u, $v, $w], [$x, $y, $z] );
310 This optimisation is only performed if the operator declared it safe to
312 do so, via the "XPI_OPERAND_ONLY_LOOK" flag.
313 If a function of the given name already exists at registration time it
315 will be left undisturbed and no new wrapper will be created. This
316 permits the same infix operator to have multiple spellings of its name;
317 for example to allow both a real Unicode and a fallback ASCII
318 transliteration of the same operator. The first registration will
319 create the wrapper function; the subsequent one will skip it because it
320 would otherwise be identical.
321 Note that when generating an optree for a wrapper function call, the
323 "new_op" hook function will be invoked with a "NULL" pointer for the
324 "SV *"-typed parse data storage, as there won't be an opporunity for
325 the "parse" hook to run in this case.
326
328 This module operates with B::Deparse in order to automatically provide
329 deparse support for infix operators. Every infix operator that is
330 implemented as a custom op (and thus has the "ppaddr" hook field set)
331 will have deparse logic added. This will allow it to deparse to either
332 the named wrapper function, or to the infix operator syntax if on a
333 "PL_infix_plugin"-enabled perl and the appropriate lexical hint is
334 enabled at the callsite.
335 In order for this to work, it is important that your custom operator is
337 not registered as a custom op using the Perl_register_custom_op()
338 function. This registration will be performed by "XS::Parse::Infix"
339 itself at the time the infix operator is registered.
340
342 • Have the entersub checker for list/list operators unwrap arrayref
343 or anon-array argument forms ("WRAPPERFUNC( \@lhs, \@rhs )" or
344 "WRAPPERFUNC( [LHS], [RHS] )").
345 • Further thoughts about how infix operators with "parse" hooks will
347 work with automatic deparse, and also how to integrate them with
348 XS::Parse::Keyword's grammar piece.
349
351 Paul Evans <leonerd@leonerd.org.uk>
352perl v5.36.1 2023-06-15 XS::Parse::Infix(3)