1LWP::UserAgent(3) User Contributed Perl Documentation LWP::UserAgent(3)
2
6 LWP::UserAgent - Web user agent class
7
9 use strict;
10 use warnings;
11 use LWP::UserAgent ();
13 my $ua = LWP::UserAgent->new(timeout => 10);
15 $ua->env_proxy;
16 my $response = $ua->get('http://example.com');
18 if ($response->is_success) {
20 print $response->decoded_content;
21 }
22 else {
23 die $response->status_line;
24 }
25 Extra layers of security (note the "cookie_jar" and
27 "protocols_allowed"):
28 use strict;
30 use warnings;
31 use HTTP::CookieJar::LWP ();
33 use LWP::UserAgent ();
34 my $jar = HTTP::CookieJar::LWP->new;
36 my $ua = LWP::UserAgent->new(
37 cookie_jar => $jar,
38 protocols_allowed => ['http', 'https'],
39 timeout => 10,
40 );
41 $ua->env_proxy;
43 my $response = $ua->get('http://example.com');
45 if ($response->is_success) {
47 print $response->decoded_content;
48 }
49 else {
50 die $response->status_line;
51 }
52
54 The LWP::UserAgent is a class implementing a web user agent.
55 LWP::UserAgent objects can be used to dispatch web requests.
56 In normal use the application creates an LWP::UserAgent object, and
58 then configures it with values for timeouts, proxies, name, etc. It
59 then creates an instance of HTTP::Request for the request that needs to
60 be performed. This request is then passed to one of the request method
61 the UserAgent, which dispatches it using the relevant protocol, and
62 returns a HTTP::Response object. There are convenience methods for
63 sending the most common request types: "get" in LWP::UserAgent, "head"
64 in LWP::UserAgent, "post" in LWP::UserAgent, "put" in LWP::UserAgent
65 and "delete" in LWP::UserAgent. When using these methods, the creation
66 of the request object is hidden as shown in the synopsis above.
67 The basic approach of the library is to use HTTP-style communication
69 for all protocol schemes. This means that you will construct
70 HTTP::Request objects and receive HTTP::Response objects even for non-
71 HTTP resources like gopher and ftp. In order to achieve even more
72 similarity to HTTP-style communications, gopher menus and file
73 directories are converted to HTML documents.
74
76 The following constructor methods are available:
77 clone
79 my $ua2 = $ua->clone;
80 Returns a copy of the LWP::UserAgent object.
82 CAVEAT: Please be aware that the clone method does not copy or clone
84 your "cookie_jar" attribute. Due to the limited restrictions on what
85 can be used for your cookie jar, there is no way to clone the
86 attribute. The "cookie_jar" attribute will be "undef" in the new object
87 instance.
88 new
90 my $ua = LWP::UserAgent->new( %options )
91 This method constructs a new LWP::UserAgent object and returns it.
93 Key/value pair arguments may be provided to set up the initial state.
94 The following options correspond to attribute methods described below:
95 KEY DEFAULT
97 ----------- --------------------
98 agent "libwww-perl/#.###"
99 conn_cache undef
100 cookie_jar undef
101 cookie_jar_class HTTP::Cookies
102 default_headers HTTP::Headers->new
103 from undef
104 local_address undef
105 max_redirect 7
106 max_size undef
107 no_proxy []
108 parse_head 1
109 protocols_allowed undef
110 protocols_forbidden undef
111 proxy {}
112 requests_redirectable ['GET', 'HEAD']
113 send_te 1
114 show_progress undef
115 ssl_opts { verify_hostname => 1 }
116 timeout 180
117 The following additional options are also accepted: If the "env_proxy"
119 option is passed in with a true value, then proxy settings are read
120 from environment variables (see "env_proxy" in LWP::UserAgent). If
121 "env_proxy" isn't provided, the "PERL_LWP_ENV_PROXY" environment
122 variable controls if "env_proxy" in LWP::UserAgent is called during
123 initialization. If the "keep_alive" option value is defined and non-
124 zero, then an "LWP::ConnCache" is set up (see "conn_cache" in
125 LWP::UserAgent). The "keep_alive" value is passed on as the
126 "total_capacity" for the connection cache.
127 "proxy" must be set as an arrayref of key/value pairs. "no_proxy" takes
129 an arrayref of domains.
130
132 The settings of the configuration attributes modify the behaviour of
133 the LWP::UserAgent when it dispatches requests. Most of these can also
134 be initialized by options passed to the constructor method.
135 The following attribute methods are provided. The attribute value is
137 left unchanged if no argument is given. The return value from each
138 method is the old attribute value.
139 agent
141 my $agent = $ua->agent;
142 $ua->agent('Checkbot/0.4 '); # append the default to the end
143 $ua->agent('Mozilla/5.0');
144 $ua->agent(""); # don't identify
145 Get/set the product token that is used to identify the user agent on
147 the network. The agent value is sent as the "User-Agent" header in the
148 requests.
149 The default is a string of the form "libwww-perl/#.###", where "#.###"
151 is substituted with the version number of this library.
152 If the provided string ends with space, the default "libwww-perl/#.###"
154 string is appended to it.
155 The user agent string should be one or more simple product identifiers
157 with an optional version number separated by the "/" character.
158 conn_cache
160 my $cache_obj = $ua->conn_cache;
161 $ua->conn_cache( $cache_obj );
162 Get/set the LWP::ConnCache object to use. See LWP::ConnCache for
164 details.
165 cookie_jar
167 my $jar = $ua->cookie_jar;
168 $ua->cookie_jar( $cookie_jar_obj );
169 Get/set the cookie jar object to use. The only requirement is that the
171 cookie jar object must implement the extract_cookies($response) and
172 add_cookie_header($request) methods. These methods will then be
173 invoked by the user agent as requests are sent and responses are
174 received. Normally this will be a HTTP::Cookies object or some
175 subclass. You are, however, encouraged to use HTTP::CookieJar::LWP
176 instead. See "BEST PRACTICES" for more information.
177 use HTTP::CookieJar::LWP ();
179 my $jar = HTTP::CookieJar::LWP->new;
181 my $ua = LWP::UserAgent->new( cookie_jar => $jar );
182 # or after object creation
184 $ua->cookie_jar( $cookie_jar );
185 The default is to have no cookie jar, i.e. never automatically add
187 "Cookie" headers to the requests.
188 If $jar contains an unblessed hash reference, a new cookie jar object
190 is created for you automatically. The object is of the class set with
191 the "cookie_jar_class" constructor argument, which defaults to
192 HTTP::Cookies.
193 $ua->cookie_jar({ file => "$ENV{HOME}/.cookies.txt" });
195 is really just a shortcut for:
197 require HTTP::Cookies;
199 $ua->cookie_jar(HTTP::Cookies->new(file => "$ENV{HOME}/.cookies.txt"));
200 As described above and in "BEST PRACTICES", you should set
202 "cookie_jar_class" to "HTTP::CookieJar::LWP" to get a safer cookie jar.
203 my $ua = LWP::UserAgent->new( cookie_jar_class => 'HTTP::CookieJar::LWP' );
205 $ua->cookie_jar({}); # HTTP::CookieJar::LWP takes no args
206 These can also be combined into the constructor, so a jar is created at
208 instantiation.
209 my $ua = LWP::UserAgent->new(
211 cookie_jar_class => 'HTTP::CookieJar::LWP',
212 cookie_jar => {},
213 );
214 credentials
216 my $creds = $ua->credentials();
217 $ua->credentials( $netloc, $realm );
218 $ua->credentials( $netloc, $realm, $uname, $pass );
219 $ua->credentials("www.example.com:80", "Some Realm", "foo", "secret");
220 Get/set the user name and password to be used for a realm.
222 The $netloc is a string of the form "<host>:<port>". The username and
224 password will only be passed to this server.
225 default_header
227 $ua->default_header( $field );
228 $ua->default_header( $field => $value );
229 $ua->default_header('Accept-Encoding' => scalar HTTP::Message::decodable());
230 $ua->default_header('Accept-Language' => "no, en");
231 This is just a shortcut for "$ua->default_headers->header( $field =>
233 $value )".
234 default_headers
236 my $headers = $ua->default_headers;
237 $ua->default_headers( $headers_obj );
238 Get/set the headers object that will provide default header values for
240 any requests sent. By default this will be an empty HTTP::Headers
241 object.
242 from
244 my $from = $ua->from;
245 $ua->from('foo@bar.com');
246 Get/set the email address for the human user who controls the
248 requesting user agent. The address should be machine-usable, as
249 defined in RFC2822 <https://tools.ietf.org/html/rfc2822>. The "from"
250 value is sent as the "From" header in the requests.
251 The default is to not send a "From" header. See "default_headers" in
253 LWP::UserAgent for the more general interface that allow any header to
254 be defaulted.
255 local_address
257 my $address = $ua->local_address;
258 $ua->local_address( $address );
259 Get/set the local interface to bind to for network connections. The
261 interface can be specified as a hostname or an IP address. This value
262 is passed as the "LocalAddr" argument to IO::Socket::INET.
263 max_redirect
265 my $max = $ua->max_redirect;
266 $ua->max_redirect( $n );
267 This reads or sets the object's limit of how many times it will obey
269 redirection responses in a given request cycle.
270 By default, the value is 7. This means that if you call "request" in
272 LWP::UserAgent and the response is a redirect elsewhere which is in
273 turn a redirect, and so on seven times, then LWP gives up after that
274 seventh request.
275 max_size
277 my $size = $ua->max_size;
278 $ua->max_size( $bytes );
279 Get/set the size limit for response content. The default is "undef",
281 which means that there is no limit. If the returned response content
282 is only partial, because the size limit was exceeded, then a
283 "Client-Aborted" header will be added to the response. The content
284 might end up longer than "max_size" as we abort once appending a chunk
285 of data makes the length exceed the limit. The "Content-Length"
286 header, if present, will indicate the length of the full content and
287 will normally not be the same as length($res->content).
288 parse_head
290 my $bool = $ua->parse_head;
291 $ua->parse_head( $boolean );
292 Get/set a value indicating whether we should initialize response
294 headers from the <head> section of HTML documents. The default is true.
295 Do not turn this off unless you know what you are doing.
296 protocols_allowed
298 my $aref = $ua->protocols_allowed; # get allowed protocols
299 $ua->protocols_allowed( \@protocols ); # allow ONLY these
300 $ua->protocols_allowed(undef); # delete the list
301 $ua->protocols_allowed(['http',]); # ONLY allow http
302 By default, an object has neither a "protocols_allowed" list, nor a
304 "protocols_forbidden" in LWP::UserAgent list.
305 This reads (or sets) this user agent's list of protocols that the
307 request methods will exclusively allow. The protocol names are case
308 insensitive.
309 For example: "$ua->protocols_allowed( [ 'http', 'https'] );" means that
311 this user agent will allow only those protocols, and attempts to use
312 this user agent to access URLs with any other schemes (like
313 "ftp://...") will result in a 500 error.
314 Note that having a "protocols_allowed" list causes any
316 "protocols_forbidden" in LWP::UserAgent list to be ignored.
317 protocols_forbidden
319 my $aref = $ua->protocols_forbidden; # get the forbidden list
320 $ua->protocols_forbidden(\@protocols); # do not allow these
321 $ua->protocols_forbidden(['http',]); # All http reqs get a 500
322 $ua->protocols_forbidden(undef); # delete the list
323 This reads (or sets) this user agent's list of protocols that the
325 request method will not allow. The protocol names are case insensitive.
326 For example: "$ua->protocols_forbidden( [ 'file', 'mailto'] );" means
328 that this user agent will not allow those protocols, and attempts to
329 use this user agent to access URLs with those schemes will result in a
330 500 error.
331 requests_redirectable
333 my $aref = $ua->requests_redirectable;
334 $ua->requests_redirectable( \@requests );
335 $ua->requests_redirectable(['GET', 'HEAD',]); # the default
336 This reads or sets the object's list of request names that
338 "redirect_ok" in LWP::UserAgent will allow redirection for. By default,
339 this is "['GET', 'HEAD']", as per RFC 2616
340 <https://tools.ietf.org/html/rfc2616>. To change to include "POST",
341 consider:
342 push @{ $ua->requests_redirectable }, 'POST';
344 send_te
346 my $bool = $ua->send_te;
347 $ua->send_te( $boolean );
348 If true, will send a "TE" header along with the request. The default is
350 true. Set it to false to disable the "TE" header for systems who can't
351 handle it.
352 show_progress
354 my $bool = $ua->show_progress;
355 $ua->show_progress( $boolean );
356 Get/set a value indicating whether a progress bar should be displayed
358 on the terminal as requests are processed. The default is false.
359 ssl_opts
361 my @keys = $ua->ssl_opts;
362 my $val = $ua->ssl_opts( $key );
363 $ua->ssl_opts( $key => $value );
364 Get/set the options for SSL connections. Without argument return the
366 list of options keys currently set. With a single argument return the
367 current value for the given option. With 2 arguments set the option
368 value and return the old. Setting an option to the value "undef"
369 removes this option.
370 The options that LWP relates to are:
372 "verify_hostname" => $bool
374 When TRUE LWP will for secure protocol schemes ensure it connects
375 to servers that have a valid certificate matching the expected
376 hostname. If FALSE no checks are made and you can't be sure that
377 you communicate with the expected peer. The no checks behaviour
378 was the default for libwww-perl-5.837 and earlier releases.
379 This option is initialized from the "PERL_LWP_SSL_VERIFY_HOSTNAME"
381 environment variable. If this environment variable isn't set; then
382 "verify_hostname" defaults to 1.
383 "SSL_ca_file" => $path
385 The path to a file containing Certificate Authority certificates.
386 A default setting for this option is provided by checking the
387 environment variables "PERL_LWP_SSL_CA_FILE" and "HTTPS_CA_FILE" in
388 order.
389 "SSL_ca_path" => $path
391 The path to a directory containing files containing Certificate
392 Authority certificates. A default setting for this option is
393 provided by checking the environment variables
394 "PERL_LWP_SSL_CA_PATH" and "HTTPS_CA_DIR" in order.
395 Other options can be set and are processed directly by the SSL Socket
397 implementation in use. See IO::Socket::SSL or Net::SSL for details.
398 The libwww-perl core no longer bundles protocol plugins for SSL. You
400 will need to install LWP::Protocol::https separately to enable support
401 for processing https-URLs.
402 timeout
404 my $secs = $ua->timeout;
405 $ua->timeout( $secs );
406 Get/set the timeout value in seconds. The default value is 180 seconds,
408 i.e. 3 minutes.
409 The request is aborted if no activity on the connection to the server
411 is observed for "timeout" seconds. This means that the time it takes
412 for the complete transaction and the "request" in LWP::UserAgent method
413 to actually return might be longer.
414 When a request times out, a response object is still returned. The
416 response will have a standard HTTP Status Code (500). This response
417 will have the "Client-Warning" header set to the value of "Internal
418 response". See the "get" in LWP::UserAgent method description below
419 for further details.
420
422 The following methods set up when requests should be passed via a proxy
423 server.
424 env_proxy
426 $ua->env_proxy;
427 Load proxy settings from *_proxy environment variables. You might
429 specify proxies like this (sh-syntax):
430 gopher_proxy=http://proxy.my.place/
432 wais_proxy=http://proxy.my.place/
433 no_proxy="localhost,example.com"
434 export gopher_proxy wais_proxy no_proxy
435 csh or tcsh users should use the "setenv" command to define these
437 environment variables.
438 On systems with case insensitive environment variables there exists a
440 name clash between the CGI environment variables and the "HTTP_PROXY"
441 environment variable normally picked up by "env_proxy". Because of
442 this "HTTP_PROXY" is not honored for CGI scripts. The "CGI_HTTP_PROXY"
443 environment variable can be used instead.
444 no_proxy
446 $ua->no_proxy( @domains );
447 $ua->no_proxy('localhost', 'example.com');
448 $ua->no_proxy(); # clear the list
449 Do not proxy requests to the given domains. Calling "no_proxy" without
451 any domains clears the list of domains.
452 proxy
454 $ua->proxy(\@schemes, $proxy_url)
455 $ua->proxy(['http', 'ftp'], 'http://proxy.sn.no:8001/');
456 # For a single scheme:
458 $ua->proxy($scheme, $proxy_url)
459 $ua->proxy('gopher', 'http://proxy.sn.no:8001/');
460 # To set multiple proxies at once:
462 $ua->proxy([
463 ftp => 'http://ftp.example.com:8001/',
464 [ 'http', 'https' ] => 'http://http.example.com:8001/',
465 ]);
466 Set/retrieve proxy URL for a scheme.
468 The first form specifies that the URL is to be used as a proxy for
470 access methods listed in the list in the first method argument, i.e.
471 "http" and "ftp".
472 The second form shows a shorthand form for specifying proxy URL for a
474 single access scheme.
475 The third form demonstrates setting multiple proxies at once. This is
477 also the only form accepted by the constructor.
478
480 Handlers are code that injected at various phases during the processing
481 of requests. The following methods are provided to manage the active
482 handlers:
483 add_handler
485 $ua->add_handler( $phase => \&cb, %matchspec )
486 Add handler to be invoked in the given processing phase. For how to
488 specify %matchspec see "Matching" in HTTP::Config.
489 The possible values $phase and the corresponding callback signatures
491 are as follows. Note that the handlers are documented in the order in
492 which they will be run, which is:
493 request_preprepare
495 request_prepare
496 request_send
497 response_header
498 response_data
499 response_done
500 response_redirect
501 request_preprepare => sub { my($request, $ua, $handler) = @_; ... }
503 The handler is called before the "request_prepare" and other
504 standard initialization of the request. This can be used to set up
505 headers and attributes that the "request_prepare" handler depends
506 on. Proxy initialization should take place here; but in general
507 don't register handlers for this phase.
508 request_prepare => sub { my($request, $ua, $handler) = @_; ... }
510 The handler is called before the request is sent and can modify the
511 request any way it see fit. This can for instance be used to add
512 certain headers to specific requests.
513 The method can assign a new request object to $_[0] to replace the
515 request that is sent fully.
516 The return value from the callback is ignored. If an exception is
518 raised it will abort the request and make the request method return
519 a "400 Bad request" response.
520 request_send => sub { my($request, $ua, $handler) = @_; ... }
522 This handler gets a chance of handling requests before they're sent
523 to the protocol handlers. It should return an HTTP::Response
524 object if it wishes to terminate the processing; otherwise it
525 should return nothing.
526 The "response_header" and "response_data" handlers will not be
528 invoked for this response, but the "response_done" will be.
529 response_header => sub { my($response, $ua, $handler) = @_; ... }
531 This handler is called right after the response headers have been
532 received, but before any content data. The handler might set up
533 handlers for data and might croak to abort the request.
534 The handler might set the "$response->{default_add_content}" value
536 to control if any received data should be added to the response
537 object directly. This will initially be false if the
538 "$ua->request()" method was called with a $content_file or
539 "$content_cb argument"; otherwise true.
540 response_data => sub { my($response, $ua, $handler, $data) = @_; ... }
542 This handler is called for each chunk of data received for the
543 response. The handler might croak to abort the request.
544 This handler needs to return a TRUE value to be called again for
546 subsequent chunks for the same request.
547 response_done => sub { my($response, $ua, $handler) = @_; ... }
549 The handler is called after the response has been fully received,
550 but before any redirect handling is attempted. The handler can be
551 used to extract information or modify the response.
552 response_redirect => sub { my($response, $ua, $handler) = @_; ... }
554 The handler is called in "$ua->request" after "response_done". If
555 the handler returns an HTTP::Request object we'll start over with
556 processing this request instead.
557 For all of these, $handler is a code reference to the handler that is
559 currently being run.
560 get_my_handler
562 $ua->get_my_handler( $phase, %matchspec );
563 $ua->get_my_handler( $phase, %matchspec, $init );
564 Will retrieve the matching handler as hash ref.
566 If $init is passed as a true value, create and add the handler if it's
568 not found. If $init is a subroutine reference, then it's called with
569 the created handler hash as argument. This sub might populate the hash
570 with extra fields; especially the callback. If $init is a hash
571 reference, merge the hashes.
572 handlers
574 $ua->handlers( $phase, $request )
575 $ua->handlers( $phase, $response )
576 Returns the handlers that apply to the given request or response at the
578 given processing phase.
579 remove_handler
581 $ua->remove_handler( undef, %matchspec );
582 $ua->remove_handler( $phase, %matchspec );
583 $ua->remove_handler(); # REMOVE ALL HANDLERS IN ALL PHASES
584 Remove handlers that match the given %matchspec. If $phase is not
586 provided, remove handlers from all phases.
587 Be careful as calling this function with %matchspec that is not
589 specific enough can remove handlers not owned by you. It's probably
590 better to use the "set_my_handler" in LWP::UserAgent method instead.
591 The removed handlers are returned.
593 set_my_handler
595 $ua->set_my_handler( $phase, $cb, %matchspec );
596 $ua->set_my_handler($phase, undef); # remove handler for phase
597 Set handlers private to the executing subroutine. Works by defaulting
599 an "owner" field to the %matchspec that holds the name of the called
600 subroutine. You might pass an explicit "owner" to override this.
601 If $cb is passed as "undef", remove the handler.
603
605 The methods described in this section are used to dispatch requests via
606 the user agent. The following request methods are provided:
607 delete
609 my $res = $ua->delete( $url );
610 my $res = $ua->delete( $url, $field_name => $value, ... );
611 This method will dispatch a "DELETE" request on the given URL.
613 Additional headers and content options are the same as for the "get" in
614 LWP::UserAgent method.
615 This method will use the DELETE() function from HTTP::Request::Common
617 to build the request. See HTTP::Request::Common for a details on how
618 to pass form content and other advanced features.
619 get
621 my $res = $ua->get( $url );
622 my $res = $ua->get( $url , $field_name => $value, ... );
623 This method will dispatch a "GET" request on the given URL. Further
625 arguments can be given to initialize the headers of the request. These
626 are given as separate name/value pairs. The return value is a response
627 object. See HTTP::Response for a description of the interface it
628 provides.
629 There will still be a response object returned when LWP can't connect
631 to the server specified in the URL or when other failures in protocol
632 handlers occur. These internal responses use the standard HTTP status
633 codes, so the responses can't be differentiated by testing the response
634 status code alone. Error responses that LWP generates internally will
635 have the "Client-Warning" header set to the value "Internal response".
636 If you need to differentiate these internal responses from responses
637 that a remote server actually generates, you need to test this header
638 value.
639 Fields names that start with ":" are special. These will not
641 initialize headers of the request but will determine how the response
642 content is treated. The following special field names are recognized:
643 ':content_file' => $filename # or $filehandle
645 ':content_cb' => \&callback
646 ':read_size_hint' => $bytes
647 If a $filename or $filehandle is provided with the ":content_file"
649 option, then the response content will be saved here instead of in the
650 response object. The $filehandle may also be an object with an open
651 file descriptor, such as a File::Temp object. If a callback is
652 provided with the ":content_cb" option then this function will be
653 called for each chunk of the response content as it is received from
654 the server. If neither of these options are given, then the response
655 content will accumulate in the response object itself. This might not
656 be suitable for very large response bodies. Only one of
657 ":content_file" or ":content_cb" can be specified. The content of
658 unsuccessful responses will always accumulate in the response object
659 itself, regardless of the ":content_file" or ":content_cb" options
660 passed in. Note that errors writing to the content file (for example
661 due to permission denied or the filesystem being full) will be reported
662 via the "Client-Aborted" or "X-Died" response headers, and not the
663 "is_success" method.
664 The ":read_size_hint" option is passed to the protocol module which
666 will try to read data from the server in chunks of this size. A
667 smaller value for the ":read_size_hint" will result in a higher number
668 of callback invocations.
669 The callback function is called with 3 arguments: a chunk of data, a
671 reference to the response object, and a reference to the protocol
672 object. The callback can abort the request by invoking die(). The
673 exception message will show up as the "X-Died" header field in the
674 response returned by the "$ua->get()" method.
675 head
677 my $res = $ua->head( $url );
678 my $res = $ua->head( $url , $field_name => $value, ... );
679 This method will dispatch a "HEAD" request on the given URL. Otherwise
681 it works like the "get" in LWP::UserAgent method described above.
682 is_protocol_supported
684 my $bool = $ua->is_protocol_supported( $scheme );
685 You can use this method to test whether this user agent object supports
687 the specified "scheme". (The "scheme" might be a string (like "http"
688 or "ftp") or it might be an URI object reference.)
689 Whether a scheme is supported is determined by the user agent's
691 "protocols_allowed" or "protocols_forbidden" lists (if any), and by the
692 capabilities of LWP. I.e., this will return true only if LWP supports
693 this protocol and it's permitted for this particular object.
694 is_online
696 my $bool = $ua->is_online;
697 Tries to determine if you have access to the Internet. Returns 1 (true)
699 if the built-in heuristics determine that the user agent is able to
700 access the Internet (over HTTP) or 0 (false).
701 See also LWP::Online.
703 mirror
705 my $res = $ua->mirror( $url, $filename );
706 This method will get the document identified by URL and store it in
708 file called $filename. If the file already exists, then the request
709 will contain an "If-Modified-Since" header matching the modification
710 time of the file. If the document on the server has not changed since
711 this time, then nothing happens. If the document has been updated, it
712 will be downloaded again. The modification time of the file will be
713 forced to match that of the server.
714 Uses "move" in File::Copy to attempt to atomically replace the
716 $filename.
717 The return value is an HTTP::Response object.
719 patch
721 # Any version of HTTP::Message works with this form:
722 my $res = $ua->patch( $url, $field_name => $value, Content => $content );
723 # Using hash or array references requires HTTP::Message >= 6.12
725 use HTTP::Request 6.12;
726 my $res = $ua->patch( $url, \%form );
727 my $res = $ua->patch( $url, \@form );
728 my $res = $ua->patch( $url, \%form, $field_name => $value, ... );
729 my $res = $ua->patch( $url, $field_name => $value, Content => \%form );
730 my $res = $ua->patch( $url, $field_name => $value, Content => \@form );
731 This method will dispatch a "PATCH" request on the given URL, with
733 %form or @form providing the key/value pairs for the fill-in form
734 content. Additional headers and content options are the same as for the
735 "get" in LWP::UserAgent method.
736 CAVEAT:
738 This method can only accept content that is in key-value pairs when
740 using HTTP::Request::Common prior to version 6.12. Any use of hash or
741 array references will result in an error prior to version 6.12.
742 This method will use the "PATCH" function from HTTP::Request::Common to
744 build the request. See HTTP::Request::Common for a details on how to
745 pass form content and other advanced features.
746 post
748 my $res = $ua->post( $url, \%form );
749 my $res = $ua->post( $url, \@form );
750 my $res = $ua->post( $url, \%form, $field_name => $value, ... );
751 my $res = $ua->post( $url, $field_name => $value, Content => \%form );
752 my $res = $ua->post( $url, $field_name => $value, Content => \@form );
753 my $res = $ua->post( $url, $field_name => $value, Content => $content );
754 This method will dispatch a "POST" request on the given URL, with %form
756 or @form providing the key/value pairs for the fill-in form content.
757 Additional headers and content options are the same as for the "get" in
758 LWP::UserAgent method.
759 This method will use the "POST" function from HTTP::Request::Common to
761 build the request. See HTTP::Request::Common for a details on how to
762 pass form content and other advanced features.
763 put
765 # Any version of HTTP::Message works with this form:
766 my $res = $ua->put( $url, $field_name => $value, Content => $content );
767 # Using hash or array references requires HTTP::Message >= 6.07
769 use HTTP::Request 6.07;
770 my $res = $ua->put( $url, \%form );
771 my $res = $ua->put( $url, \@form );
772 my $res = $ua->put( $url, \%form, $field_name => $value, ... );
773 my $res = $ua->put( $url, $field_name => $value, Content => \%form );
774 my $res = $ua->put( $url, $field_name => $value, Content => \@form );
775 This method will dispatch a "PUT" request on the given URL, with %form
777 or @form providing the key/value pairs for the fill-in form content.
778 Additional headers and content options are the same as for the "get" in
779 LWP::UserAgent method.
780 CAVEAT:
782 This method can only accept content that is in key-value pairs when
784 using HTTP::Request::Common prior to version 6.07. Any use of hash or
785 array references will result in an error prior to version 6.07.
786 This method will use the "PUT" function from HTTP::Request::Common to
788 build the request. See HTTP::Request::Common for a details on how to
789 pass form content and other advanced features.
790 request
792 my $res = $ua->request( $request );
793 my $res = $ua->request( $request, $content_file );
794 my $res = $ua->request( $request, $content_cb );
795 my $res = $ua->request( $request, $content_cb, $read_size_hint );
796 This method will dispatch the given $request object. Normally this will
798 be an instance of the HTTP::Request class, but any object with a
799 similar interface will do. The return value is an HTTP::Response
800 object.
801 The "request" method will process redirects and authentication
803 responses transparently. This means that it may actually send several
804 simple requests via the "simple_request" in LWP::UserAgent method
805 described below.
806 The request methods described above; "get" in LWP::UserAgent, "head" in
808 LWP::UserAgent, "post" in LWP::UserAgent and "mirror" in LWP::UserAgent
809 will all dispatch the request they build via this method. They are
810 convenience methods that simply hide the creation of the request object
811 for you.
812 The $content_file, $content_cb and $read_size_hint all correspond to
814 options described with the "get" in LWP::UserAgent method above. Note
815 that errors writing to the content file (for example due to permission
816 denied or the filesystem being full) will be reported via the
817 "Client-Aborted" or "X-Died" response headers, and not the "is_success"
818 method.
819 You are allowed to use a CODE reference as "content" in the request
821 object passed in. The "content" function should return the content
822 when called. The content can be returned in chunks. The content
823 function will be invoked repeatedly until it return an empty string to
824 signal that there is no more content.
825 simple_request
827 my $request = HTTP::Request->new( ... );
828 my $res = $ua->simple_request( $request );
829 my $res = $ua->simple_request( $request, $content_file );
830 my $res = $ua->simple_request( $request, $content_cb );
831 my $res = $ua->simple_request( $request, $content_cb, $read_size_hint );
832 This method dispatches a single request and returns the response
834 received. Arguments are the same as for the "request" in
835 LWP::UserAgent described above.
836 The difference from "request" in LWP::UserAgent is that
838 "simple_request" will not try to handle redirects or authentication
839 responses. The "request" in LWP::UserAgent method will, in fact,
840 invoke this method for each simple request it sends.
841
843 The following methods will be invoked as requests are processed. These
844 methods are documented here because subclasses of LWP::UserAgent might
845 want to override their behaviour.
846 get_basic_credentials
848 # This checks wantarray and can either return an array:
849 my ($user, $pass) = $ua->get_basic_credentials( $realm, $uri, $isproxy );
850 # or a string that looks like "user:pass"
851 my $creds = $ua->get_basic_credentials($realm, $uri, $isproxy);
852 This is called by "request" in LWP::UserAgent to retrieve credentials
854 for documents protected by Basic or Digest Authentication. The
855 arguments passed in is the $realm provided by the server, the $uri
856 requested and a "boolean flag" to indicate if this is authentication
857 against a proxy server.
858 The method should return a username and password. It should return an
860 empty list to abort the authentication resolution attempt. Subclasses
861 can override this method to prompt the user for the information. An
862 example of this can be found in "lwp-request" program distributed with
863 this library.
864 The base implementation simply checks a set of pre-stored member
866 variables, set up with the "credentials" in LWP::UserAgent method.
867 prepare_request
869 $request = $ua->prepare_request( $request );
870 This method is invoked by "simple_request" in LWP::UserAgent. Its task
872 is to modify the given $request object by setting up various headers
873 based on the attributes of the user agent. The return value should
874 normally be the $request object passed in. If a different request
875 object is returned it will be the one actually processed.
876 The headers affected by the base implementation are; "User-Agent",
878 "From", "Range" and "Cookie".
879 progress
881 my $prog = $ua->progress( $status, $request_or_response );
882 This is called frequently as the response is received regardless of how
884 the content is processed. The method is called with $status "begin" at
885 the start of processing the request and with $state "end" before the
886 request method returns. In between these $status will be the fraction
887 of the response currently received or the string "tick" if the fraction
888 can't be calculated.
889 When $status is "begin" the second argument is the HTTP::Request
891 object, otherwise it is the HTTP::Response object.
892 redirect_ok
894 my $bool = $ua->redirect_ok( $prospective_request, $response );
895 This method is called by "request" in LWP::UserAgent before it tries to
897 follow a redirection to the request in $response. This should return a
898 true value if this redirection is permissible. The
899 $prospective_request will be the request to be sent if this method
900 returns true.
901 The base implementation will return false unless the method is in the
903 object's "requests_redirectable" list, false if the proposed
904 redirection is to a "file://..." URL, and true otherwise.
905
907 The default settings can get you up and running quickly, but there are
908 settings you can change in order to make your life easier.
909 Handling Cookies
911 You are encouraged to install Mozilla::PublicSuffix and use
912 HTTP::CookieJar::LWP as your cookie jar. HTTP::CookieJar::LWP provides
913 a better security model matching that of current Web browsers when
914 Mozilla::PublicSuffix is installed.
915 use HTTP::CookieJar::LWP ();
917 my $jar = HTTP::CookieJar::LWP->new;
919 my $ua = LWP::UserAgent->new( cookie_jar => $jar );
920 See "cookie_jar" for more information.
922 Managing Protocols
924 "protocols_allowed" gives you the ability to allow arbitrary protocols.
925 my $ua = LWP::UserAgent->new(
927 protocols_allowed => [ 'http', 'https' ]
928 );
929 This will prevent you from inadvertently following URLs like
931 "file:///etc/passwd". See "protocols_allowed".
932 "protocols_forbidden" gives you the ability to deny arbitrary
934 protocols.
935 my $ua = LWP::UserAgent->new(
937 protocols_forbidden => [ 'file', 'mailto', 'ssh', ]
938 );
939 This can also prevent you from inadvertently following URLs like
941 "file:///etc/passwd". See "protocols_forbidden".
942
944 See LWP for a complete overview of libwww-perl5. See lwpcook and the
945 scripts lwp-request and lwp-download for examples of usage.
946 See HTTP::Request and HTTP::Response for a description of the message
948 objects dispatched and received. See HTTP::Request::Common and
949 HTML::Form for other ways to build request objects.
950 See WWW::Mechanize and WWW::Search for examples of more specialized
952 user agents based on LWP::UserAgent.
953
955 Copyright 1995-2009 Gisle Aas.
956 This library is free software; you can redistribute it and/or modify it
958 under the same terms as Perl itself.
959perl v5.38.0 2023-07-25 LWP::UserAgent(3)