1Genlex(3)                        OCaml library                       Genlex(3)
2
3
4

NAME

6       Genlex - A generic lexical analyzer.
7

Module

9       Module   Genlex
10

Documentation

12       Module Genlex
13        : sig end
14
15
16       A generic lexical analyzer.
17
18       This  module implements a simple 'standard' lexical analyzer, presented
19       as a function from character streams to token  streams.  It  implements
20       roughly  the  lexical conventions of OCaml, but is parameterized by the
21       set of keywords of your language.
22
23       Example: a lexer suitable for a desk  calculator  is  obtained  by  let
24       lexer = make_lexer ["+";"-";"*";"/";"let";"="; ( ; ) ]
25
26       The  associated  parser  would  be a function from token stream to, for
27       instance, int , and would have rules such as:
28
29
30       let rec parse_expr = parser | [< n1 = parse_atom; n2 =  parse_remainder
31       n1 >] -> n2 and parse_atom = parser | [< 'Int n >] -> n | [< 'Kwd ( ; n
32       = parse_expr; 'Kwd ) >] -> n and parse_remainder n1 = parser | [<  'Kwd
33       + ; n2 = parse_expr >] -> n1+n2 | [< >] -> n1
34
35       One  should  notice  that  the use of the parser keyword and associated
36       notation for streams are only available through camlp4 extensions. This
37       means  that  one  has  to preprocess its sources e. g. by using the -pp
38       command-line switch of the compilers.
39
40
41
42
43
44       type token =
45        | Kwd of string
46        | Ident of string
47        | Int of int
48        | Float of float
49        | String of string
50        | Char of char
51
52
53       The type of tokens. The lexical classes are: Int and Float for  integer
54       and  floating-point  numbers;  String  for string literals, enclosed in
55       double quotes; Char for character literals, enclosed in single  quotes;
56       Ident for identifiers (either sequences of letters, digits, underscores
57       and quotes, or sequences of 'operator characters' such as + , * , etc);
58       and Kwd for keywords (either identifiers or single 'special characters'
59       such as ( , } , etc).
60
61
62
63       val make_lexer : string list -> char Stream.t -> token Stream.t
64
65       Construct the lexer function. The first argument is the  list  of  key‐
66       words.  An identifier s is returned as Kwd s if s belongs to this list,
67       and as Ident s otherwise.  A special character s is returned as  Kwd  s
68       if  s  belongs  to  this  list,  and  cause  a lexical error (exception
69       Stream.Error with the offending lexeme  as  its  parameter)  otherwise.
70       Blanks  and  newlines  are skipped. Comments delimited by (* and *) are
71       skipped as well, and can  be  nested.  A  Stream.Failure  exception  is
72       raised if end of stream is unexpectedly reached.
73
74
75
76
77
78OCamldoc                          2018-07-14                         Genlex(3)
Impressum