1JSON::Tokenize(3)     User Contributed Perl Documentation    JSON::Tokenize(3)
2
3
4

NAME

6       JSON::Tokenize - tokenize a string containing JSON
7

SYNOPSIS

9           use JSON::Tokenize ':all';
10           my $input = '{"tuttie":["fruity", true, 100]}';
11           my $token = tokenize_json ($input);
12           print_tokens ($token, 0);
13
14           sub print_tokens
15           {
16               my ($token, $depth) = @_;
17               while ($token) {
18                   my $start = tokenize_start ($token);
19                   my $end = tokenize_end ($token);
20                   my $type = tokenize_type ($token);
21                   print "   " x $depth;
22                   my $value = substr ($input, $start, $end - $start);
23                   print ">>$value<< has type $type\n";
24                   my $child = tokenize_child ($token);
25                   if ($child) {
26                       print_tokens ($child, $depth+1);
27                   }
28                   my $next = tokenize_next ($token);
29                   $token = $next;
30               }
31           }
32
33       This outputs
34
35           >>{"tuttie":["fruity", true, 100]}<< has type object
36              >>"tuttie"<< has type string
37              >>:<< has type colon
38              >>["fruity", true, 100]<< has type array
39                 >>"fruity"<< has type string
40                 >>,<< has type comma
41                 >>true<< has type literal
42                 >>,<< has type comma
43                 >>100<< has type number
44

VERSION

46       This documents version 0.57 of JSON::Tokenize corresponding to git
47       commit 42e099ed38a3e85daed46f53bf367fa2b828e152
48       <https://github.com/benkasminbullock/JSON-
49       Parse/commit/42e099ed38a3e85daed46f53bf367fa2b828e152> released on Thu
50       Jul 9 07:16:33 2020 +0900.
51

DESCRIPTION

53       This is a module for tokenizing a JSON string. It breaks the string
54       into individual tokens without creating any Perl structures. Thus it
55       can be used for tasks such as picking out or searching through parts of
56       a large JSON structure without storing each part of the entire
57       structure as individual Perl variables in memory.
58
59       This module is an experimental part of JSON::Parse and its interface is
60       likely to change. The tokenizing functions are currently written in a
61       very primitive way.
62

FUNCTIONS

64   tokenize_json
65           my $token = tokenize_json ($json);
66
67   tokenize_next
68           my $next = tokenize_next ($token);
69
70       Walk the tree of tokens.
71
72   tokenize_child
73           my $child = tokenize_child ($child);
74
75       Walk the tree of tokens.
76
77   tokenize_start
78           my $start = tokenize_start ($token);
79
80       Get the start of the token as a byte offset from the start of the
81       string. Note this is a byte offset not a character offset.
82
83   tokenize_end
84           my $end = tokenize_end ($token);
85
86       Get the end of the token as a byte offset from the start of the string.
87       Note this is a byte offset not a character offset.
88
89   tokenize_type
90           my $type = tokenize_type ($token);
91
92       Get the type of the token as a string. The possible return values are
93
94           "invalid",
95           "initial state",
96           "string",
97           "number",
98           "literal",
99           "object",
100           "array",
101           "unicode escape"
102
103   tokenize_text
104           my $text = tokenize_text ($json, $token);
105
106       Given a token $token from this parsing and the JSON in $json, return
107       the text which corresponds to the token. This is a convenience function
108       written in Perl which uses "tokenize_start" and "tokenize_end" and
109       "substr" to get the string from $json.
110

AUTHOR

112       Ben Bullock, <bkb@cpan.org>
113

115       This package and associated files are copyright (C) 2016-2020 Ben
116       Bullock.
117
118       You can use, copy, modify and redistribute this package and associated
119       files under the Perl Artistic Licence or the GNU General Public
120       Licence.
121
122
123
124perl v5.32.0                      2020-07-28                 JSON::Tokenize(3)
Impressum