1Hijk(3pm) User Contributed Perl Documentation Hijk(3pm)
2
6 Hijk - Fast & minimal low-level HTTP client
7
9 A simple GET request:
10 use Hijk ();
12 my $res = Hijk::request({
13 method => "GET",
14 host => "example.com",
15 port => "80",
16 path => "/flower",
17 query_string => "color=red"
18 });
19 if (exists $res->{error} and $res->{error} & Hijk::Error::TIMEOUT) {
21 die "Oh noes we had some sort of timeout";
22 }
23 die "Expecting an 'OK' response" unless $res->{status} == 200;
25 say $res->{body};
27 A POST request, you have to manually set the appropriate headers, URI
29 escape your values etc.
30 use Hijk ();
32 use URI::Escape qw(uri_escape);
33 my $res = Hijk::request({
35 method => "POST",
36 host => "example.com",
37 port => "80",
38 path => "/new",
39 head => [ "Content-Type" => "application/x-www-form-urlencoded" ],
40 query_string => "type=flower&bucket=the%20one%20out%20back",
41 body => "description=" . uri_escape("Another flower, let's hope it's exciting"),
42 });
43 die "Expecting an 'OK' response" unless $res->{status} == 200;
45
47 Hijk is a fast & minimal low-level HTTP client intended to be used
48 where you control both the client and the server, e.g. for talking to
49 some internal service from a frontend user-facing web application.
50 It is "NOT" a general HTTP user agent, it doesn't support redirects,
52 proxies, SSL and any number of other advanced HTTP features like (in
53 roughly descending order of feature completeness) LWP::UserAgent,
54 WWW::Curl, HTTP::Tiny, HTTP::Lite or Furl. This library is basically
55 one step above manually talking HTTP over sockets.
56 Having said that it's lightning fast and extensively used in production
58 at Booking.com <https://www.booking.com> where it's used as the go-to
59 transport layer for talking to internal services. It uses non-blocking
60 sockets and correctly handles all combinations of connect/read timeouts
61 and other issues you might encounter from various combinations of parts
62 of your system going down or becoming otherwise unavailable.
63
65 "Hijk::request" is the only function you should use. It (or anything
66 else in this package for that matter) is not exported, so you have to
67 use the fully qualified name.
68 It takes a "HashRef" of arguments and either dies or returns a
70 "HashRef" as a response.
71 The "HashRef" argument to it must contain some of the key-value pairs
73 from the following list. The value for "host" and "port" are mandatory,
74 but others are optional with default values listed below.
75 protocol => "HTTP/1.1", # (or "HTTP/1.0")
77 host => ...,
78 port => ...,
79 connect_timeout => undef,
80 read_timeout => undef,
81 read_length => 10240,
82 method => "GET",
83 path => "/",
84 query_string => "",
85 head => [],
86 body => "",
87 socket_cache => \%Hijk::SOCKET_CACHE, # (undef to disable, or \my %your_socket_cache)
88 on_connect => undef, # (or sub { ... })
89 parse_chunked => 0,
90 head_as_array => 0,
91 no_default_host_header => 1,
92 Notice how Hijk does not take a full URI string as input, you have to
94 specify the individual parts of the URL. Users who need to parse an
95 existing URI string to produce a request should use the URI module to
96 do so.
97 The value of "head" is an "ArrayRef" of key-value pairs instead of a
99 "HashRef", this way you can decide in which order the headers are sent,
100 and you can send the same header name multiple times. For example:
101 head => [
103 "Content-Type" => "application/json",
104 "X-Requested-With" => "Hijk",
105 ]
106 Will produce these request headers:
108 Content-Type: application/json
110 X-Requested-With: Hijk
111 In addition Hijk will provide a "Host" header for you by default with
113 the "host" value you pass to "request()". To suppress this (e.g. to
114 send custom "Host" requests) pass a true value to the
115 "no_default_host_header" option and provide your own "Host" header in
116 the "head" "ArrayRef" (or don't, if you want to construct a "Host"-less
117 request knock yourself out...).
118 Hijk doesn't escape any values for you, it just passes them through as-
120 is. You can easily produce invalid requests if e.g. any of these
121 strings contain a newline, or aren't otherwise properly escaped.
122 The value of "connect_timeout" or "read_timeout" is in floating point
124 seconds, and is used as the time limit for connecting to the host, and
125 reading the response back from it, respectively. The default value for
126 both is "undef", meaning no timeout limit. If you don't supply these
127 timeouts and the host really is unreachable or slow, we'll reach the
128 TCP timeout limit before returning some other error to you.
129 The default "protocol" is "HTTP/1.1", but you can also specify
131 "HTTP/1.0". The advantage of using "HTTP/1.1" is support for keep-
132 alive, which matters a lot in environments where the connection setup
133 represents non-trivial overhead. Sometimes that overhead is negligible
134 (e.g. on Linux talking to an nginx on the local network), and keeping
135 open connections down and reducing complexity is more important, in
136 those cases you can either use "HTTP/1.0", or specify "Connection:
137 close" in the request, but just using "HTTP/1.0" is an easy way to
138 accomplish the same thing.
139 By default we will provide a "socket_cache" for you which is a global
141 singleton that we maintain keyed on "join($;, $$, $host, $port)".
142 Alternatively you can pass in "socket_cache" hash of your own which
143 we'll use as the cache. To completely disable the cache pass in
144 "undef".
145 The optional "on_connect" callback is intended to be used for you to
147 figure out from production traffic what you should set the
148 "connect_timeout". I.e. you can start a timer when you call
149 "Hijk::request()" that you end when "on_connect" is called, that's how
150 long it took us to get a connection. If you start another timer in that
151 callback that you end when "Hijk::request()" returns to you that'll
152 give you how long it took to send/receive data after we constructed the
153 socket, i.e. it'll help you to tweak your "read_timeout". The
154 "on_connect" callback is provided with no arguments, and is called in
155 void context.
156 We have experimental support for parsing chunked responses encoding.
158 historically Hijk didn't support this at all and if you wanted to use
159 it with e.g. nginx you had to add "chunked_transfer_encoding off" to
160 the nginx config file.
161 Since you may just want to do that instead of having Hijk do more work
163 to parse this out with a more complex and experimental codepath you
164 have to explicitly enable it with "parse_chunked". Otherwise Hijk will
165 die when it encounters chunked responses. The "parse_chunked" option
166 may be turned on by default in the future.
167 The return value is a "HashRef" representing a response. It contains
169 the following key-value pairs.
170 proto => :Str
172 status => :StatusCode
173 body => :Str
174 head => :HashRef (or :ArrayRef with "head_as_array")
175 error => :PositiveInt
176 error_message => :Str
177 errno_number => :Int
178 errno_string => :Str
179 For example, to send a request to
181 "http://example.com/flower?color=red", pass the following parameters:
182 my $res = Hijk::request({
184 host => "example.com",
185 port => "80",
186 path => "/flower",
187 query_string => "color=red"
188 });
189 die "Response is not 'OK'" unless $res->{status} == 200;
190 Notice that you do not need to put the leading "?" character in the
192 "query_string". You do, however, need to properly "uri_escape" the
193 content of "query_string".
194 Again, Hijk doesn't escape any values for you, so these values MUST be
196 properly escaped before being passed in, unless you want to issue
197 invalid requests.
198 By default the "head" in the response is a "HashRef" rather then an
200 "ArrayRef". This makes it easier to retrieve specific header fields,
201 but it means that we'll clobber any duplicated header names with the
202 most recently seen header value. To get the returned headers as an
203 "ArrayRef" instead specify "head_as_array".
204 If you want to fiddle with the "read_length" value it controls how much
206 we "POSIX::read($fd, $buf, $read_length)" at a time.
207 We currently don't support servers returning a http body without an
209 accompanying "Content-Length" header; bodies MUST have a
210 "Content-Length" or we won't pick them up.
211
213 If we had a recoverable error we'll include an "error" key whose value
214 is a bitfield that you can check against Hijk::Error::* constants.
215 Those are:
216 Hijk::Error::CONNECT_TIMEOUT
218 Hijk::Error::READ_TIMEOUT
219 Hijk::Error::TIMEOUT
220 Hijk::Error::CANNOT_RESOLVE
221 Hijk::Error::REQUEST_SELECT_ERROR
222 Hijk::Error::REQUEST_WRITE_ERROR
223 Hijk::Error::REQUEST_ERROR
224 Hijk::Error::RESPONSE_READ_ERROR
225 Hijk::Error::RESPONSE_BAD_READ_VALUE
226 Hijk::Error::RESPONSE_ERROR
227 In addition we might return "error_message", "errno_number" and
229 "errno_string" keys, see the discussion of "Hijk::Error::REQUEST_*" and
230 "Hijk::Error::RESPONSE_*" errors below.
231 The "Hijk::Error::TIMEOUT" constant is the same as
233 "Hijk::Error::CONNECT_TIMEOUT | Hijk::Error::READ_TIMEOUT". It's there
234 for convenience so you can do:
235 .. if exists $res->{error} and $res->{error} & Hijk::Error::TIMEOUT;
237 Instead of the more verbose:
239 .. if exists $res->{error} and $res->{error} & (Hijk::Error::CONNECT_TIMEOUT | Hijk::Error::READ_TIMEOUT)
241 We'll return "Hijk::Error::CANNOT_RESOLVE" if we can't
243 "gethostbyname()" the host you've provided.
244 If we fail to do a "select()" or "write()" during when sending the
246 response we'll return "Hijk::Error::REQUEST_SELECT_ERROR" or
247 "Hijk::Error::REQUEST_WRITE_ERROR", respectively. Similarly to
248 "Hijk::Error::TIMEOUT" the "Hijk::Error::REQUEST_ERROR" constant is a
249 union of these two, and any other request errors we might add in the
250 future.
251 When we're getting the response back we'll return
253 "Hijk::Error::RESPONSE_READ_ERROR" when we can't "read()" the response,
254 and "Hijk::Error::RESPONSE_BAD_READ_VALUE" when the value we got from
255 "read()" is 0. The "Hijk::Error::RESPONSE_ERROR" constant is a union of
256 these two and any other response errors we might add in the future.
257 Some of these "Hijk::Error::REQUEST_*" and "Hijk::Error::RESPONSE_*"
259 errors are re-thrown errors from system calls. In that case we'll also
260 pass along "error_message" which is a short human readable error
261 message about the error, as well as "errno_number" & "errno_string",
262 which are "$!+0" and "$!" at the time we had the error.
263 Hijk might encounter other errors during the course of the request and
265 WILL call "die" if that happens, so if you don't want your program to
266 stop when a request like that fails wrap it in "eval".
267 Having said that the point of the "Hijk::Error::*" interface is that
269 all errors that happen during normal operation, i.e. making valid
270 requests against servers where you can have issues like timeouts,
271 network blips or the server thread on the other end being suddenly kill
272 -9'd should be caught, categorized and returned in a structural way by
273 Hijk.
274 We're not currently aware of any issues that occur in such normal
276 operations that aren't classified as a "Hijk::Error::*", and if we find
277 new issues that fit the criteria above we'll likely just make a new
278 "Hijk::Error::*" for it.
279 We're just not trying to guarantee that the library can never "die",
281 and aren't trying to catch truly exceptional issues like e.g. "fcntl()"
282 failing on a valid socket.
283
285 Kang-min Liu <gugod@gugod.org>
286 AEvar Arnfjoerd` Bjarmason <avar@cpan.org>
288 Borislav Nikolov <jack@sofialondonmoskva.com>
290 Damian Gryski <damian@gryski.com>
292
294 Copyright (c) 2013- Kang-min Liu "<gugod@gugod.org>".
295
297 The MIT License
298
300 BECAUSE THIS SOFTWARE IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
301 FOR THE SOFTWARE, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT
302 WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER
303 PARTIES PROVIDE THE SOFTWARE "AS IS" WITHOUT WARRANTY OF ANY KIND,
304 EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
305 WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE
306 ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE SOFTWARE IS WITH
307 YOU. SHOULD THE SOFTWARE PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL
308 NECESSARY SERVICING, REPAIR, OR CORRECTION.
309 IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
311 WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
312 REDISTRIBUTE THE SOFTWARE AS PERMITTED BY THE ABOVE LICENCE, BE LIABLE
313 TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL, OR
314 CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE
315 SOFTWARE (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING
316 RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A
317 FAILURE OF THE SOFTWARE TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF
318 SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH
319 DAMAGES.
320perl v5.32.0 2020-07-28 Hijk(3pm)