1fennel-api(3) Fennel's Lua API fennel-api(3)
2
6 fennel-api - Fennel's Lua API
7
9 The fennel module provides the following functions for use when embed‐
10 ding Fennel in a Lua program. If you're writing a pure Fennel program
11 or working on a system that already has Fennel support, you probably
12 don't need this.
13 Only the fennel module is part of the public API. The other modules
15 are implementation details subject to change. Most functions will er‐
16 ror upon failure.
17 Any time a function takes an options table argument, that table will
19 usually accept these fields:
20 • allowedGlobals: a sequential table of strings of the names of globals
22 which the compiler will allow references to. Set to false to disable
23 checks. Defaults to the contents of the env table, if provided, or
24 the current environment.
25 • correlate: when this is set, Fennel attempts to emit Lua where the
27 line numbers match up with the Fennel input code; useful for situa‐
28 tion where code that isn't under your control will print the stack
29 traces.
30 • useMetadata (since 0.3.0): enables or disables metadata, allowing use
32 of the ,doc repl command. Intended for development purposes (see
33 performance note); defaults to true for REPL only.
34 • requireAsInclude (since 0.3.0): Alias any static require calls to the
36 include special, embedding the module code inline in the compiled
37 output. If the module name isn't a string literal that is resolvable
38 at compile time it falls back to require at runtime. Can be used to
39 embed both Fennel and Lua modules.
40 • env: an environment table in which to run the code; see the Lua manu‐
42 al.
43 • compilerEnv: an environment table in which to run compiler-scoped
45 code for macro definitions and eval-compiler calls. Internal Fennel
46 functions such as list, sym, etc. will be exposed in addition to
47 this table. Defaults to a table containing limited known-safe glob‐
48 als. Pass _G to disable sandboxing.
49 • unfriendly: disable friendly compiler/parser error messages.
51 • plugins: list of compiler plugins.
53 • error-pinpoint: a list of two strings indicating what to wrap compile
55 errors in
56 You can pass the string "_COMPILER" as the value for env; it will cause
58 the code to be run/compiled in a context which has all compiler-scoped
59 values available. This can be useful for macro modules or compiler
60 plugins.
61 Note that only the fennel module is part of the public API. The other
63 modules (fennel.utils, fennel.compiler, etc) should be considered com‐
64 piler internals subject to change.
65 If you are embedding Fennel in a context where ANSI escape codes are
67 not interpreted, you can set error-pinpoint to false to disable the
68 highlighting of compiler and parse errors.
69
71 fennel.repl([options])
72 Takes these additional options:
74 • readChunk(): a function that when called, returns a string of source
76 code. The empty is string or nil is used as the end of source mark‐
77 er.
78 • pp: a pretty-printer function to apply on values (default: fen‐
80 nel.view).
81 • onValues(values): a function that will be called on all returned top
83 level values.
84 • onError(errType, err, luaSource): a function that will be called on
86 each error. errType is a string with the type of error, can be ei‐
87 ther, 'parse', 'compile', 'runtime', or 'lua'. err is the error mes‐
88 sage, and luaSource is the source of the generated lua code.
89 By default, metadata will be enabled and you can view function signa‐
91 tures and docstrings with the ,doc command in the REPL.
92
94 local result = fennel.eval(str[, options[, ...]])
95 The options table may also contain:
97 • filename: override the filename that Lua thinks the code came from.
99 Additional arguments beyond options are passed to the code and avail‐
101 able as ....
102
104 local result = fennel.dofile(filename[, options[, ...]])
105 Additional arguments beyond options are passed to the code and avail‐
107 able as ....
108
110 require("fennel").install().dofile("main.fnl")
111 This is the equivalent of this code:
113 local fennel = require("fennel")
115 table.insert(package.loaders or package.searchers, fennel.searcher)
116 fennel.dofile("main.fnl") -- require calls in main.fnl can load fennel modules
117 Normally Lua's require function only loads modules written in Lua, but
119 you can install fennel.searcher into package.searchers (or in Lua 5.1
120 package.loaders) to teach it how to load Fennel code.
121 If you would rather change some of the options you can use fennel.make‐
123 Searcher(options) to get a searcher function that's equivalent to fen‐
124 nel.searcher but overrides the default options table.
125 The require function is different from fennel.dofile in that it search‐
127 es the directories in fennel.path for .fnl files matching the module
128 name, and also in that it caches the loaded value to return on subse‐
129 quent calls, while fennel.dofile will reload each time. The behavior
130 of fennel.path mirrors that of Lua's package.path. There is also a
131 fennel.macro-path which is used to look up macro modules.
132 If you install Fennel into package.searchers then you can use the re‐
134 pl's ,reload mod command to reload modules that have been loaded with
135 require.
136
138 The compiler sandbox makes it so that the module system is also isolat‐
139 ed from the rest of the system, so the above require calls will not
140 work from inside macros. However, there is a separate fennel.macro-
141 searchers table which can be used to allow different modules to be
142 loaded inside macros. By default it includes a searcher to load sand‐
143 boxed Fennel modules and a searcher to load sandboxed Lua modules, but
144 if you disable the compiler sandbox you may want to replace these with
145 searchers which can load arbitrary modules.
146 The default fennel.macro-searchers functions also cannot load C mod‐
148 ules. Here's an example of some code which would allow that to work:
149 table.insert(fennel["macro-searchers"], function(module_name)
151 local filename = fennel["search-module"](module_name, package.cpath)
152 if filename then
153 local func = "luaopen_" .. module_name
154 return function() return package.loadlib(filename, func) end, filename
155 end
156 end)
157 Macro searchers store loaded macro modules in the fennel.macro-loaded
159 table which works the same as package.loaded but for macro modules.
160
162 The fennel.traceback function works like Lua's debug.traceback func‐
163 tion, except it tracks line numbers from Fennel code correctly.
164 If you are working on an application written in Fennel, you can over‐
166 ride the default traceback function to replace it with Fennel's:
167 debug.traceback = fennel.traceback
169 Note that some systems print stack traces from C, which will not be af‐
171 fected.
172
174 print(fennel.searchModule("my.mod", package.path))
175 If you just want to find the file path that a module would resolve to
177 without actually loading it, you can use fennel.searchModule. The
178 first argument is the module name, and the second argument is the path
179 string to search. If none is provided, it defaults to Fennel's own
180 path.
181 Returns nil if the module is not found on the path.
183
185 local lua = fennel.compileString(str[, options])
186 Accepts indent as a string in options causing output to be indented us‐
188 ing that string, which should contain only whitespace if provided. Un‐
189 like the other functions, the compile functions default to performing
190 no global checks, though you can pass in an allowedGlobals table in op‐
191 tions to enable it.
192 Accepts filename in options as in fennel.eval.
194
196 This is useful when streaming data into the compiler to allow you to
197 avoid loading all the code into a single string in one go.
198 local lua = fennel.compileStream(strm[, options])
200 Accepts indent and filename in options as per above.
202
204 The ast here can be gotten from fennel.parser.
205 local lua = fennel.compile(ast[, options])
207 Accepts indent and filename in options as per above.
209
211 The fennel.parser function returns a stateful iterator function. If a
212 form was successfully read, it returns true followed by the AST node.
213 Returns nil when it reaches the end. Raises an error if it can't parse
214 the input.
215 local parse = fennel.parser(text)
217 local ok, ast = assert(parse()) -- just get the first form
218 -- Or use in a for loop
220 for ok, ast in parse do
221 if ok then
222 print(fennel.view(ast))
223 end
224 end
225 The first argument can either be a string or a function that returns
227 one byte at a time. It takes two optional arguments; a filename and a
228 table of options. Supported options are both booleans that default to
229 false:
230 • unfriendly: disable enhanced parse error reporting
232 • comments: include comment nodes in AST
234 • plugins: (since 1.2.0) An optional list of compiler plugins.
236 The list of common options at the top of this document do not apply
238 here.
239
241 The AST returned by the parser consists of data structures representing
242 the code. Passing AST nodes to the fennel.view function will give you
243 a string which should round-trip thru the parser to give you the same
244 data back. The same is true with tostring, except it does not work
245 with non-sequence tables.
246 The fennel.ast-source function takes an AST node and returns a table
248 with source data around filename, line number, et in it, if possible.
249 Some AST nodes cannot provide this data, for instance numbers, strings,
250 and booleans, or symbols constructed within macros using the sym func‐
251 tion instead of backtick.
252 AST nodes can be any of these types:
254 list
256 A list represents a call to function/macro, or destructuring multiple
257 return values in a binding context. It's represented as a table which
258 can be identified using the fennel.list? predicate function or con‐
259 structed using fennel.list which takes any number of arguments for the
260 contents of the list.
261 Note that lists are compile-time constructs in Fennel. They do not ex‐
263 ist at runtime, except in such cases as the compiler is in use at run‐
264 time.
265 The list also contains these keys indicating where it was defined:
267 filename, line, col, endcol, bytestart, and byteend. This data is used
268 for stack traces and for pinpointing compiler error messages. Note
269 that column numbers are based on character count, which does not always
270 correspond to visual columns; for instance "วัด" is three characters
271 but only two visual columns.
272 sequence/kv table
274 These are table literals in Fennel code produced by square brackets
275 (sequences) or curly brackets (kv tables). Sequences can be identified
276 using the fennel.sequence? function and constructed using fennel.se‐
277 quence. There is no predicate or constructor for kv tables; any table
278 which is not one of the other types is assumed to be one of these.
279 At runtime there is no difference between sequences and kv tables which
281 use monotonically increasing integer keys, but the parser is able to
282 distinguish between them to improve error reporting.
283 Sequences and kv tables have their source data in filename, line, etc
285 keys of their metatable. The metatable for kv tables also includes a
286 keys sequence which tells you which order the keys appeared originally,
287 since kv tables are unordered and there would otherwise be no way to
288 reconstruct this information.
289 symbol
291 Symbols typically represent identifiers in Fennel code. Symbols can be
292 identified with fennel.sym? and constructed with fennel.sym which takes
293 a string name as its first argument and a source data table as the sec‐
294 ond. Symbols are represented as tables which store their source data
295 (filename, line, col, etc) in fields on themselves. Unlike the other
296 tables in the AST, they do not represent collections; they are used as
297 scalar types.
298 Symbols can refer not just directly to locals, but also to table refer‐
300 ences like tbl.x for field lookup or access.channel:deny for method in‐
301 vocation. The fennel.multi-sym? function will return a table contain‐
302 ing the segments if the symbol if it is one of these, or nil otherwise.
303 Note: nil is not a valid AST; code that references nil will have the
305 symbol named "nil" which unfortunately prints in a way that is visually
306 indistinguishable from actual nil.
307 The fennel.sym-char? function will tell you if a given character is al‐
309 lowed to be used in the name of a symbol.
310 vararg
312 This is a special type of symbol-like construct (...) indicating func‐
313 tions using a variable number of arguments. Its meaning is the same as
314 in Lua. It's identified with fennel.varg? and constructed with fen‐
315 nel.varg.
316 number/string/boolean
318 These are literal types defined by Lua. They cannot carry source data.
319 comment
321 By default, ASTs will omit comments. However, when the :comment field
322 is set in the parser options, comments will be included in the parsed
323 values. They are identified using fennel.comment? and constructed us‐
324 ing the fennel.comment function. They are represented as tables that
325 have source data as fields inside them.
326 In most data context, comments just get included inline in a list or
328 sequence. However, in a kv table, this cannot be done, because kv ta‐
329 bles must have balanced key/value pairs, and including comments inline
330 would imbalance these or cause keys to be considered as values and vice
331 versa. So the comments are stored on the comments field of metatable
332 instead, keyed by the key or value they were attached to.
333
335 The fennel.view function takes any Fennel data and turns it into a rep‐
336 resentation suitable for feeding back to Fennel's parser. In addition
337 to tables, strings, numbers, and booleans, it can produce reasonable
338 output from ASTs that come from the parser. It will emit an unreadable
339 placeholder for coroutines, compiled functions, and userdata, which
340 cannot be understood by the parser.
341 print(fennel.view({abc=123}[, options])
343 {:abc 123}
344 The list of common options at the top of this document do not apply
346 here; instead these options are accepted:
347 • one-line? (default: false) keep the output string as a one-liner
349 • depth (number, default: 128) limit how many levels to go (default:
351 128)
352 • detect-cycles? (default: true) don't try to traverse a looping table
354 • metamethod? (default: true) use the __fennelview metamethod if found
356 • empty-as-sequence? (default: false) render empty tables as []
358 • line-length (number, default: 80) length of the line at which multi-
360 line output for tables is forced
361 • byte-escape (function) If present, overrides default behavior of es‐
363 caping special characters in decimal format (e.g. <ESC> -> \027).
364 Called with the signature (byte-escape byte view-opts), where byte is
365 the char code for a special character
366 • escape-newlines? (default: false) emit strings with \n instead of
368 newline
369 • prefer-colon? (default: false) emit strings in colon notation when
371 possible
372 • utf8? (default true) whether to use utf8 module to compute string
374 lengths
375 • max-sparse-gap (integer, default 10) maximum gap to fill in with nils
377 in sparse sequential tables.
378 • preprocess (function) if present, called on x (and recursively on
380 each value in x), and the result is used for pretty printing; takes
381 the same arguments as fennel.view
382 All options can be set to {:once some-value} to force their value to be
384 some-value but only for the current level. After that, such option is
385 reset to its default value. Alternatively, {:once value :after other-
386 value} can be used, with the difference that after first use, the op‐
387 tions will be set to other-value instead of the default value.
388 You can set a __fennelview metamethod on a table to override its seri‐
390 alization behavior. It should take the table being serialized as its
391 first argument, a function as its second argument, options table as
392 third argument, and current amount of indentation as its last argument:
393 (fn [t view options indent] ...)
395 view function contains a pretty printer that can be used to serialize
397 elements stored within the table being serialized. If your metamethod
398 produces indented representation, you should pass indent parameter to
399 view increased by the amount of additional indentation you've intro‐
400 duced. This function has the same interface as __fennelview
401 metamethod, but in addition accepts colon-string? as last argument. If
402 colon? is true, strings will be printed as colon-strings when possible,
403 and if its value is false, strings will be always printed in double
404 quotes. If omitted or nil will default to value of :prefer-colon? op‐
405 tion.
406 options table contains options described above, and also visible-cycle?
408 function, that takes a table being serialized, detects and saves infor‐
409 mation about possible reachable cycle. Should be used in __fennelview
410 to implement cycle detection.
411 __fennelview metamethod should always return a table of correctly in‐
413 dented lines when producing multi-line output, or a string when always
414 returning single-line item. fennel.view will transform your data
415 structure to correct multi-line representation when needed. There's no
416 need to concatenate table manually ever - fennel.view will apply gener‐
417 al rules for your data structure, depending on current options. By de‐
418 fault multiline output is produced only when inner data structures con‐
419 tains newlines, or when returning table of lines as single line results
420 in width greater than line-size option.
421 Multi-line representation can be forced by returning two values from
423 __fennelview - a table of indented lines as first value, and true as
424 second value, indicating that multi-line representation should be
425 forced.
426 There's no need to incorporate indentation beyond needed to correctly
428 align elements within the printed representation of your data struc‐
429 ture. For example, if you want to print a multi-line table, like this:
430 @my-table[1
432 2
433 3]
434 __fennelview should return a sequence of lines:
436 ["@my-table[1"
438 " 2"
439 " 3]"]
440 Note, since we've introduced inner indent string of length 10, when
442 calling view function from within __fennelview metamethod, in order to
443 keep inner tables indented correctly, indent must be increased by this
444 amount of extra indentation.
445 Here's an implementation of such pretty-printer for an arbitrary se‐
447 quential table:
448 (fn pp-doc-example [t view options indent]
450 (let [lines (icollect [i v (ipairs t)]
451 (let [v (view v options (+ 10 indent))]
452 (if (= i 1) v
453 (.. " " v))))]
454 (doto lines
455 (tset 1 (.. "@my-table[" (or (. lines 1) "")))
456 (tset (length lines) (.. (. lines (length lines)) "]")))))
457 Setting table's __fennelview metamethod to this function will provide
459 correct results regardless of nesting:
460 >> {:my-table (setmetatable [[1 2 3 4 5]
462 {:smalls [6 7 8 9 10 11 12]
463 :bigs [500 1000 2000 3000 4000]}]
464 {:__fennelview pp-doc-example})
465 :normal-table [{:c [1 2 3] :d :some-data} 4]}
466 {:my-table @my-table[[1 2 3 4 5]
467 {:bigs [500 1000 2000 3000 4000]
468 :smalls [6 7 8 9 10 11 12]}]
469 :normal-table [{:c [1 2 3] :d "some-data"} 4]}
470 Note that even though we've only indented inner elements of our table
472 with 10 spaces, the result is correctly indented in terms of outer ta‐
473 ble, and inner tables also remain indented correctly.
474 When using the :preprocess option or __fennelview method, avoid modify‐
476 ing any tables in-place in the passed function. Since Lua tables are
477 mutable and passed in without copying, any modification done in these
478 functions will be visible outside of fennel.view.
479 Using :byte-escape to override the special character escape format is
481 intended for use-cases where it's known that the output will be con‐
482 sumed by something other than Lua/Fennel, and may result in output that
483 Fennel can no longer parse. For example, to force the use of hex es‐
484 capes:
485 (print (fennel.view {:clear-screen "\027[H\027[2J"}
487 {:byte-escape #(: "\\x%2x" :format $)}))
488 ;; > {:clear-screen "\x1b[H\x1b[2J"}
489 While Lua 5.2+ supports hex escapes, PUC Lua 5.1 does not, so compiling
491 this with Fennel later would result in an incorrect escape code in Lua
492 5.1.
493
495 (Since 0.3.0)
496 When running a REPL or using compile/eval with metadata enabled, each
498 function declared with fn or λ/lambda will use the created function as
499 a key on fennel.metadata to store the function's arglist and (if pro‐
500 vided) docstring. The metadata table is weakly-referenced by key, so
501 each function's metadata will be garbage collected along with the func‐
502 tion itself.
503 You can work with the API to view or modify this metadata yourself, or
505 use the ,doc repl command to view function documentation.
506 In addition to direct access to the metadata tables, you can use the
508 following methods:
509 • fennel.metadata:get(func, key): get a value from a function's metada‐
511 ta
512 • fennel.metadata:set(func, key, val): set a metadata value
514 • fennel.metadata:setall(func, key1, val1, key2, val2, ...): set pairs
516 • fennel.doc(func, fnName): print formatted documentation for function
518 using name. Utilized by the ,doc command, name is whatever symbol
519 you operate on that's bound to the function.
520 local greet = fennel.eval('(λ greet [name] "Say hello" (print "Hello," name))',
522 {useMetadata = true})
523 fennel.metadata[greet]
525 -- > {"fnl/docstring" = "Say hello", "fnl/arglist" = ["name"]}
526 fennel.doc(greet, "greet")
528 -- > (greet name)
529 -- > Say hello
530 fennel.metadata:set(greet, "fnl/docstring", "Say hello!!!")
532 fennel.doc(greet, "greet!")
533 --> (greet! name)
534 --> Say hello!!!
535 Metadata performance note
537 Enabling metadata in the compiler/eval/REPL will cause every function
538 to store a new table containing the function's arglist and docstring in
539 the metadata table, weakly referenced by the function itself as a key.
540 This may have a performance impact in some applications due to the ex‐
542 tra allocations and garbage collection associated with dynamic function
543 creation. The impact hasn't been benchmarked, but enabling metadata is
544 currently recommended for development purposes only.
545
547 If you're writing a tool which performs syntax highlighting or some
548 other operations on Fennel code, the fennel.syntax function can provide
549 you with data about what forms and keywords to treat specially.
550 local syntax = fennel.syntax()
552 print(fennel.view(syntax["icollect"]))
553 --> {:binding-form? true :body-form? true :macro? true}
554 The table has string keys and table values. Each entry will have one
556 of "macro?", "global?", or "special?" set to true indicating what type
557 it is. Globals can also have "function?" set to true. Macros and spe‐
558 cials can have "binding-form?" set to true indicating it accepts a []
559 argument which introduces new locals, and/or a "body-form?" indicating
560 whether it should be indented with two spaces instead of being indented
561 like a function call. They can also have a "define?" key indicating
562 whether it introduces a new top-level identifier like local or fn.
563
565 This isn't Fennel-specific, but the loadCode function takes a string of
566 Lua code along with an optional environment table and filename string,
567 and returns a function for the loaded code which will run inside that
568 environment, in a way that's portable across any Lua 5.1+ version.
569 local f = fennel.loadCode(luaCode, { x = y }, "myfile.lua")
571
573 This function does a best effort detection of the Lua VM environment
574 hosting Fennel. Useful for displaying an "About" dialog in your Fennel
575 app that matches the REPL and --version CLI flag.
576 (fennel.runtime-version)
578 print(fennel.runtimeVersion())
580 -- > Fennel 1.0.0 on PUC Lua 5.4
581 The fennel.version field will give you the version of just Fennel it‐
583 self.
584 (since 1.3.1)
586 If an optional argument is given, returns version information as a ta‐
588 ble:
589 (fennel.runtime-version :as-table)
591 ;; > {:fennel "1.3.1" :lua "PUC Lua 5.4"}
592
594 Fennel's plugin system is extremely experimental and exposes internals
595 of the compiler in ways that no other part of the compiler does. It
596 should be considered unstable; changes to the compiler in future ver‐
597 sions are likely to break plugins, and each plugin should only be as‐
598 sumed to work with specific versions of the compiler that they're test‐
599 ed against. The backwards-compatibility guarantees of the rest of Fen‐
600 nel do not apply to plugins.
601 Compiler plugins allow the functionality of the compiler to be extended
603 in various ways. A plugin is a module containing various functions in
604 fields named after different compiler extension points. When the com‐
605 piler hits an extension point, it will call each plugin's function for
606 that extension point, if provided, with various arguments; usually the
607 AST in question and the scope table. Each plugin function should nor‐
608 mally do side effects and return nil or error out. If a function re‐
609 turns non-nil, it will cause the rest of the plugins for a given event
610 to be skipped.
611 • symbol-to-expression
613 • call
615 • do
617 • fn
619 • destructure
621 • parse-error
623 • assert-compile
625 The destructure extension point is different because instead of just
627 taking ast and scope it takes a from which is the AST for the value be‐
628 ing destructured and a to AST which is the AST for the form being de‐
629 structured to. This is most commonly a symbol but can be a list or a
630 table.
631 The parse-error and assert-compile hooks can be used to override how
633 fennel behaves down to the parser and compiler levels. Possible use-
634 cases include building atop fennel.view to serialize data with EDN
635 (https://clojure.github.io/clojure/clojure.edn-api.html)-style tagging,
636 or manipulating external s-expression-based syntax, such as tree-sitter
637 queries (https://tree-sitter.github.io/tree-sitter/using-parsers#query-
638 syntax).
639 The scope argument is a table containing all the compiler's information
641 about the current scope. Most of the tables here look up values in
642 their parent scopes if they do not contain a key.
643 Plugins can also contain repl commands. If your plugin module has a
645 field with a name beginning with "repl-command-" then that function
646 will be available as a comma command from within a repl session. It
647 will be called with a table for the repl session's environment, a func‐
648 tion which will read the next form from stdin, a function which is used
649 to print normal values, and one which is used to print errors.
650 (local fennel (require :fennel)
652 (fn locals [env read on-values on-error scope]
653 "Print all locals in repl session scope."
654 (on-values [(fennel.view env.___replLocals___)]))
655 {:repl-command-locals locals}
657 $ fennel --plugin locals-plugin.fnl
659 Welcome to Fennel 0.8.0 on Lua 5.4!
660 Use ,help to see available commands.
661 >> (local x 4)
662 nil
663 >> (local abc :xyz)
664 nil
665 >> ,locals
666 {
667 :abc "xyz"
668 :x 4
669 }
670 The docstring of the function will be used as its summary in the
672 ",help" command listing. Unlike other plugin hook fields, only the
673 first plugin to provide a repl command will be used.
674 Activation
676 Plugins are activated by passing the --plugin argument on the command
677 line, which should be a path to a Fennel file containing a module that
678 has some of the functions listed above. If you're using the compiler
679 programmatically, you can include a :plugins table in the options table
680 to most compiler entry point functions.
681 Your plugin should contain a :versions table which contains a list of
683 strings indicating every version of Fennel which you have tested it
684 with. You should also have a :name field with the plugin's name.
685
687 Fennel Maintainers.
688fennel 1.3.1 2023-07-05 fennel-api(3)