1HTTP::Proxy(3) User Contributed Perl Documentation HTTP::Proxy(3)
2
6 HTTP::Proxy - A pure Perl HTTP proxy
7
9 use HTTP::Proxy;
10 # initialisation
12 my $proxy = HTTP::Proxy->new( port => 3128 );
13 # alternate initialisation
15 my $proxy = HTTP::Proxy->new;
16 $proxy->port( 3128 ); # the classical accessors are here!
17 # this is a MainLoop-like method
19 $proxy->start;
20
22 This module implements an HTTP proxy, using an HTTP::Daemon to accept
23 client connections, and an LWP::UserAgent to ask for the requested
24 pages.
25 The most interesting feature of this proxy object is its ability to
27 filter the HTTP requests and responses through user-defined filters.
28 Once the proxy is created, with the new() method, it is possible to
30 alter its behaviour by adding so-called "filters." This is done by the
31 push_filter() method. Once the filter is ready to run, it can be
32 launched, with the start() method. This method does not normally return
33 until the proxy is killed or otherwise stopped.
34 An important thing to note is that the proxy is (except when running
36 the "NoFork" engine) a forking proxy: it doesn't support passing
37 information between child processes, and you can count on reliable
38 information passing only during a single HTTP connection (request +
39 response).
40
42 You can alter the way the default HTTP::Proxy works by plugging
43 callbacks (filter objects, actually) at different stages of the
44 request/response handling.
45 When a request is received by the HTTP::Proxy object, it is filtered
47 through a standard filter that transforms the request according to RFC
48 2616 (by adding the "Via:" header, and other transformations). This is
49 the default, bare minimum behaviour.
50 The response is also filtered in the same manner. There is a total of
52 four filter chains: "request-headers", "request-body",
53 "response-headers" and "response-body".
54 You can add your own filters to the default ones with the push_filter()
56 method. The method pushes a filter on the appropriate filter stack.
57 $proxy->push_filter( response => $filter );
59 The headers/body category is determined by the base class of the
61 filter. There are two base classes for filters, which are
62 HTTP::Proxy::HeaderFilter and HTTP::Proxy::BodyFilter (the names are
63 self-explanatory). See the documentation of those two classes to find
64 out how to write your own header and body filters.
65 The named parameter is used to determine the request/response part.
67 It is possible to push the same filter on the request and response
69 stacks, as in the following example:
70 $proxy->push_filter( request => $filter, response => $filter );
72 If several filters match the message, they will be applied in the order
74 they were pushed on their filter stack.
75 Named parameters can be used to create the match routine. They are:
77 method - the request method
79 scheme - the URI scheme
80 host - the URI authority (host:port)
81 path - the URI path
82 query - the URI query string
83 mime - the MIME type (for a response-body filter)
84 The filters are applied only when all the the parameters match the
86 request or the response. All these named parameters have default
87 values, which are:
88 method => 'OPTIONS,GET,HEAD,POST,PUT,DELETE,TRACE,CONNECT'
90 scheme => 'http'
91 host => ''
92 path => ''
93 query => ''
94 mime => 'text/*'
95 The "mime" parameter is a glob-like string, with a required "/"
97 character and a "*" as a wildcard. Thus, "*/*" matches all responses,
98 and "" those with no "Content-Type:" header. To match any repines (with
99 or without a "Content-Type:" header), use "undef".
100 The "mime" parameter is only meaningful with the "response-body" filter
102 stack. It is ignored if passed to any other filter stack.
103 The "method" and "scheme" parameters are strings consisting of comma-
105 separated values. The "host" and "path" parameters are regular
106 expressions.
107 A match routine is compiled by the proxy and used to check if a
109 particular request or response must be filtered through a particular
110 filter.
111 It is also possible to push several filters on the same stack with the
113 same match subroutine:
114 # convert italics to bold
116 $proxy->push_filter(
117 mime => 'text/html',
118 response => HTTP::Proxy::BodyFilter::tags->new(),
119 response => HTTP::Proxy::BodyFilter::simple->new(
120 sub { ${ $_[1] } =~ s!(</?)i>!$1b>!ig }
121 )
122 );
123 For more details regarding the creation of new filters, check the
125 HTTP::Proxy::HeaderFilter and HTTP::Proxy::BodyFilter documentation.
126 Here's an example of subclassing a base filter class:
128 # fixes a common typo ;-)
130 # but chances are that this will modify a correct URL
131 {
132 package FilterPerl;
133 use base qw( HTTP::Proxy::BodyFilter );
134 sub filter {
136 my ( $self, $dataref, $message, $protocol, $buffer ) = @_;
137 $$dataref =~ s/PERL/Perl/g;
138 }
139 }
140 $proxy->push_filter( response => FilterPerl->new() );
141 Other examples can be found in the documentation for
143 HTTP::Proxy::HeaderFilter, HTTP::Proxy::BodyFilter,
144 HTTP::Proxy::HeaderFilter::simple, HTTP::Proxy::BodyFilter::simple.
145 # a simple anonymiser
147 # see eg/anonymiser.pl for the complete code
148 $proxy->push_filter(
149 mime => undef,
150 request => HTTP::Proxy::HeaderFilter::simple->new(
151 sub { $_[1]->remove_header(qw( User-Agent From Referer Cookie )) },
152 ),
153 response => HTTP::Proxy::HeaderFilter::simple->new(
154 sub { $_[1]->remove_header(qw( Set-Cookie )); },
155 )
156 );
157 IMPORTANT: If you use your own LWP::UserAgent, you must install it
159 before your calls to push_filter(), otherwise the match method will
160 make wrong assumptions about the schemes your agent supports.
161 NOTE: It is likely that possibility of changing the agent or the daemon
163 may disappear in future versions.
164
166 Constructor and initialisation
167 new()
168 The new() method creates a new HTTP::Proxy object. All attributes
169 can be passed as parameters to replace the default.
170 Parameters that are not HTTP::Proxy attributes will be ignored and
172 passed to the chosen HTTP::Proxy::Engine object.
173 init()
175 init() initialise the proxy without starting it. It is usually not
176 needed.
177 This method is called by start() if needed.
179 push_filter()
181 The push_filter() method is used to add filters to the proxy. It
182 is fully described in section FILTERS.
183 Accessors and mutators
185 HTTP::Proxy class has several accessors and mutators.
186 Called with arguments, the accessor returns the current value. Called
188 with a single argument, it sets the current value and returns the
189 previous one, in case you want to keep it.
190 If you call a read-only accessor with a parameter, this parameter will
192 be ignored.
193 The defined accessors are (in alphabetical order):
195 agent
197 The LWP::UserAgent object used internally to connect to remote
198 sites.
199 chunk
201 The chunk size for the LWP::UserAgent callbacks.
202 client_socket (read-only)
204 The socket currently connected to the client. Mostly useful in
205 filters.
206 client_headers
208 This attribute holds a reference to the client headers set up by
209 LWP::UserAgent ("Client-Aborted", "Client-Bad-Header-Line",
210 "Client-Date", "Client-Junk", "Client-Peer", "Client-Request-Num",
211 "Client-Response-Num", "Client-SSL-Cert-Issuer",
212 "Client-SSL-Cert-Subject", "Client-SSL-Cipher",
213 "Client-SSL-Warning", "Client-Transfer-Encoding",
214 "Client-Warning").
215 They are removed by the filter HTTP::Proxy::HeaderFilter::standard
217 from the request and response objects received by the proxy.
218 If a filter (such as a SSL certificate verification filter) need to
220 access them, it must do it through this accessor.
221 conn (read-only)
223 The number of connections processed by this HTTP::Proxy instance.
224 daemon
226 The HTTP::Daemon object used to accept incoming connections. (You
227 usually never need this.)
228 engine
230 The HTTP::Proxy::Engine object that manages the child processes.
231 hop_headers
233 This attribute holds a reference to the hop-by-hop headers
234 ("Connection", "Keep-Alive", "Proxy-Authenticate",
235 "Proxy-Authorization", "TE", "Trailers", "Transfer-Encoding",
236 "Upgrade").
237 They are removed by the filter HTTP::Proxy::HeaderFilter::standard
239 from the request and response objects received by the proxy.
240 If a filter (such as a proxy authorisation filter) need to access
242 them, it must do it through this accessor.
243 host
245 The proxy HTTP::Daemon host (default: 'localhost').
246 This means that by default, the proxy answers only to clients on
248 the local machine. You can pass a specific interface address or
249 ""/"undef" for any interface.
250 This default prevents your proxy to be used as an anonymous proxy
252 by script kiddies.
253 known_methods( @groups ) (read-only)
255 This method returns all HTTP (and extensions to HTTP) known to
256 "HTTP::Proxy". Methods are grouped by type. Known method groups
257 are: "HTTP", "WebDAV" and "DeltaV".
258 Called with an empty list, this method will return all known
260 methods. This method is case-insensitive, and will carp() if an
261 unknown group name is passed.
262 logfh
264 A filehandle to a logfile (default: *STDERR).
265 logmask( [$mask] )
267 Be verbose in the logs (default: "NONE").
268 Here are the various elements that can be added to the mask (their
270 values are powers of 2, starting from 0 and listed here in
271 ascending order):
272 NONE - Log only errors
274 PROXY - Proxy information
275 STATUS - Requested URL, response status and total number
276 of connections processed
277 PROCESS - Subprocesses information (fork, wait, etc.)
278 SOCKET - Information about low-level sockets
279 HEADERS - Full request and response headers are sent along
280 FILTERS - Filter information
281 DATA - Data received by the filters
282 CONNECT - Data transmitted by the CONNECT method
283 ENGINE - Engine information
284 ALL - Log all of the above
285 If you only want status and process information, you can use:
287 $proxy->logmask( STATUS | PROCESS );
289 Note that all the logging constants are not exported by default,
291 but by the ":log" tag. They can also be exported one by one.
292 loop (read-only)
294 Internal. False when the main loop is about to be broken.
295 max_clients
297 maxchild
298 The maximum number of child process the HTTP::Proxy object will
299 spawn to handle client requests (default: depends on the engine).
300 This method is currently delegated to the HTTP::Proxy::Engine
302 object.
303 "maxchild" is deprecated and will disappear.
305 max_connections
307 maxconn
308 The maximum number of TCP connections the proxy will accept before
309 returning from start(). 0 (the default) means never stop accepting
310 connections.
311 "maxconn" is deprecated.
313 Note: "max_connections" will be deprecated soon, for two reasons:
315 1) it is more of an HTTP::Proxy::Engine attribute, 2) not all
316 engines will support it.
317 max_keep_alive_requests
319 maxserve
320 The maximum number of requests the proxy will serve in a single
321 connection. (same as "MaxRequestsPerChild" in Apache)
322 "maxserve" is deprecated.
324 port
326 The proxy HTTP::Daemon port (default: 8080).
327 request
329 The request originally received by the proxy from the user-agent,
330 which will be modified by the request filters.
331 response
333 The response received from the origin server by the proxy. It is
334 normally "undef" until the proxy actually receives the beginning of
335 a response from the origin server.
336 If one of the request filters sets this attribute, it "short-
338 circuits" the request/response scheme, and the proxy will return
339 this response (which is NOT filtered through the response filter
340 stacks) instead of the expected origin server response. This is
341 useful for caching (though Squid does it much better) and proxy
342 authentication, for example.
343 stash
345 The stash is a hash where filters can store data to share between
346 them.
347 The stash() method can be used to set the whole hash (with a HASH
349 reference). To access individual keys simply do:
350 $proxy->stash( 'bloop' );
352 To set it, type:
354 $proxy->stash( bloop => 'owww' );
356 It's also possibly to get a reference to the stash:
358 my $s = $filter->proxy->stash();
360 $s->{bang} = 'bam';
361 # $proxy->stash( 'bang' ) will now return 'bam'
363 Warning: since the proxy forks for each TCP connection, the data is
365 only shared between filters in the same child process.
366 timeout
368 The timeout used by the internal LWP::UserAgent (default: 60).
369 url (read-only)
371 The url where the proxy can be reached.
372 via The content of the Via: header. Setting it to an empty string will
374 prevent its addition. (default: "$hostname (HTTP::Proxy/$VERSION)")
375 x_forwarded_for
377 If set to a true value, the proxy will send the "X-Forwarded-For:"
378 header. (default: true)
379 Connection handling methods
381 start()
382 This method works like Tk's "MainLoop": you hand over control to
383 the HTTP::Proxy object you created and configured.
384 If "maxconn" is not zero, start() will return after accepting at
386 most that many connections. It will return the total number of
387 connexions.
388 serve_connections()
390 This is the internal method used to handle each new TCP connection
391 to the proxy.
392 Other methods
394 log( $level, $prefix, $message )
395 Adds $message at the end of "logfh", if $level matches "logmask".
396 The log() method also prints a timestamp.
397 The output looks like:
399 [Thu Dec 5 12:30:12 2002] ($$) $prefix: $message
401 where $$ is the current process's id.
403 If $message is a multiline string, several log lines will be
405 output, each line starting with $prefix.
406 is_protocol_supported( $scheme )
408 Returns a boolean indicating if $scheme is supported by the proxy.
409 This method is only used internally.
411 It is essential to allow HTTP::Proxy users to create "pseudo-
413 schemes" that LWP doesn't know about, but that one of the proxy
414 filters can handle directly. New schemes are added as follows:
415 $proxy->init(); # required to get an agent
417 $proxy->agent->protocols_allowed(
418 [ @{ $proxy->agent->protocols_allowed }, 'myhttp' ] );
419 new_connection()
421 Increase the proxy's TCP connections counter. Only used by
422 HTTP::Proxy::Engine objects.
423 Apache-like attributes
425 HTTP::Proxy has several Apache-like attributes that control the way the
426 HTTP and TCP connections are handled.
427 The following attributes control the TCP connection. They are passed to
429 the underlying HTTP::Proxy::Engine, which may (or may not) use them to
430 change its behaviour.
431 start_servers
433 Number of child process to fork at the beginning.
434 max_clients
436 Maximum number of concurrent TCP connections (i.e. child
437 processes).
438 max_requests_per_child
440 Maximum number of TCP connections handled by the same child
441 process.
442 min_spare_servers
444 Minimum number of inactive child processes.
445 max_spare_servers
447 Maximum number of inactive child processes.
448 Those attributes control the HTTP connection:
450 keep_alive
452 Support for keep alive HTTP connections.
453 max_keep_alive_requests
455 Maximum number of HTTP connections within a single TCP connection.
456 keep_alive_timeout
458 Timeout for keep-alive connection.
459
461 No symbols are exported by default. The ":log" tag exports all the
462 logging constants.
463
465 This module does not work under Windows, but I can't see why, and do
466 not have a development platform under that system. Patches and
467 explanations very welcome.
468 I guess it is because fork() is not well supported.
470 $proxy->maxchild(0);
472 However, David Fishburn says:
474 This did not work for me under WinXP - ActiveState Perl 5.6, but it
475 DOES work on WinXP ActiveState Perl 5.8.
476 Several people have tried to help, but we haven't found a way to make
478 it work correctly yet.
479 As from version 0.16, the default engine is
481 HTTP::Proxy::Engine::NoFork. Let me know if it works better.
482
484 HTTP::Proxy::Engine, HTTP::Proxy::BodyFilter,
485 HTTP::Proxy::HeaderFilter, the examples in eg/.
486
488 Philippe "BooK" Bruhat, <book@cpan.org>.
489 There is also a mailing-list: http-proxy@mongueurs.net for general
491 discussion about HTTP::Proxy.
492
494 Many people helped me during the development of this module, either on
495 mailing-lists, IRC or over a beer in a pub...
496 So, in no particular order, thanks to the libwww-perl team for such a
498 terrific suite of modules, perl-qa (tips for testing), the French Perl
499 Mongueurs (for code tricks, beers and encouragements) and my growing
500 user base... ";-)"
501 I'd like to particularly thank Dan Grigsby, who's been using
503 HTTP::Proxy since 2003 (before the filter classes even existed). He is
504 apparently making a living from a product based on HTTP::Proxy. Thanks
505 a lot for your confidence in my work!
506
508 Copyright 2002-2015, Philippe Bruhat.
509
511 This module is free software; you can redistribute it or modify it
512 under the same terms as Perl itself.
513perl v5.36.0 2023-01-20 HTTP::Proxy(3)