1Lexing(3)                        OCaml library                       Lexing(3)
2
3
4

NAME

6       Lexing - The run-time library for lexers generated by ocamllex.
7

Module

9       Module   Lexing
10

Documentation

12       Module Lexing
13        : sig end
14
15
16       The run-time library for lexers generated by ocamllex .
17
18
19
20
21
22
23
24
25       === Positions ===
26
27
28       type position = {
29        pos_fname : string ;
30        pos_lnum : int ;
31        pos_bol : int ;
32        pos_cnum : int ;
33        }
34
35
36       A value of type position describes a point in a source file.  pos_fname
37       is the file name; pos_lnum is the line number; pos_bol is the offset of
38       the  beginning  of the line (number of characters between the beginning
39       of the file and the beginning of the line); pos_cnum is the  offset  of
40       the  position  (number  of characters between the beginning of the file
41       and the position).
42
43       See the documentation of type lexbuf for information about how the lex‐
44       ing engine will manage positions.
45
46
47
48
49       val dummy_pos : position
50
51       A  value  of  type position , guaranteed to be different from any valid
52       position.
53
54
55
56
57
58       === Lexer buffers ===
59
60
61       type lexbuf = {
62        refill_buff : lexbuf -> unit ;
63
64       mutable lex_buffer : string ;
65
66       mutable lex_buffer_len : int ;
67
68       mutable lex_abs_pos : int ;
69
70       mutable lex_start_pos : int ;
71
72       mutable lex_curr_pos : int ;
73
74       mutable lex_last_pos : int ;
75
76       mutable lex_last_action : int ;
77
78       mutable lex_eof_reached : bool ;
79
80       mutable lex_mem : int array ;
81
82       mutable lex_start_p : position ;
83
84       mutable lex_curr_p : position ;
85        }
86
87
88       The type of lexer buffers. A lexer buffer is the argument passed to the
89       scanning functions defined by the generated scanners.  The lexer buffer
90       holds the current state of the scanner, plus a function to  refill  the
91       buffer from the input.
92
93       At  each token, the lexing engine will copy lex_curr_p to lex_start_p ,
94       then change the pos_cnum field of lex_curr_p by updating  it  with  the
95       number  of  characters  read since the start of the lexbuf .  The other
96       fields are left unchanged by the lexing engine.  In order to keep  them
97       accurate,  they must be initialised before the first use of the lexbuf,
98       and updated by the relevant lexer actions (i.e. at each end of line  --
99       see also new_line ).
100
101
102
103
104       val from_channel : Pervasives.in_channel -> lexbuf
105
106       Create  a lexer buffer on the given input channel.  Lexing.from_channel
107       inchan returns a lexer buffer which reads from the input channel inchan
108       , at the current reading position.
109
110
111
112
113       val from_string : string -> lexbuf
114
115       Create a lexer buffer which reads from the given string. Reading starts
116       from the first character in the string. An  end-of-input  condition  is
117       generated when the end of the string is reached.
118
119
120
121
122       val from_function : (string -> int -> int) -> lexbuf
123
124       Create  a  lexer  buffer with the given function as its reading method.
125       When the scanner needs more characters, it will call  the  given  func‐
126       tion,  giving  it  a  character  string s and a character count n . The
127       function should put n characters or less in s , starting  at  character
128       number  0, and return the number of characters provided. A return value
129       of 0 means end of input.
130
131
132
133
134
135       === Functions for lexer semantic actions ===
136
137
138       === The following functions can be called from the semantic actions  of
139       lexer  definitions  (the  ML  code enclosed in braces that computes the
140       value returned by lexing functions). They give access to the  character
141       string  matched  by the regular expression associated with the semantic
142       action. These functions must be applied to the argument lexbuf,  which,
143       in  the code generated by ocamllex, is bound to the lexer buffer passed
144       to the parsing function. ===
145
146
147       val lexeme : lexbuf -> string
148
149
150       Lexing.lexeme lexbuf returns the string matched by the regular  expres‐
151       sion.
152
153
154
155
156       val lexeme_char : lexbuf -> int -> char
157
158
159       Lexing.lexeme_char  lexbuf  i returns character number i in the matched
160       string.
161
162
163
164
165       val lexeme_start : lexbuf -> int
166
167
168       Lexing.lexeme_start lexbuf returns the offset in the  input  stream  of
169       the  first character of the matched string.  The first character of the
170       stream has offset 0.
171
172
173
174
175       val lexeme_end : lexbuf -> int
176
177
178       Lexing.lexeme_end lexbuf returns the offset in the input stream of  the
179       character following the last character of the matched string. The first
180       character of the stream has offset 0.
181
182
183
184
185       val lexeme_start_p : lexbuf -> position
186
187       Like lexeme_start , but return a complete position instead of  an  off‐
188       set.
189
190
191
192
193       val lexeme_end_p : lexbuf -> position
194
195       Like lexeme_end , but return a complete position instead of an offset.
196
197
198
199
200       val new_line : lexbuf -> unit
201
202       Update the lex_curr_p field of the lexbuf to reflect the start of a new
203       line.  You can call this function in the semantic action  of  the  rule
204       that matches the end-of-line character.
205
206
207
208
209
210       === Miscellaneous functions ===
211
212
213       val flush_input : lexbuf -> unit
214
215       Discard the contents of the buffer and reset the current position to 0.
216       The next use of the lexbuf will trigger a refill.
217
218
219
220
221
222
223OCamldoc                          2010-01-29                         Lexing(3)