1HTTP::Cache::TransparenUts(e3r)Contributed Perl DocumentHaTtTiPo:n:Cache::Transparent(3)
2
3
4

NAME

6       HTTP::Cache::Transparent - Cache the result of http get-requests
7       persistently.
8

SYNOPSIS

10         use LWP::Simple;
11         use HTTP::Cache::Transparent;
12
13         HTTP::Cache::Transparent::init( {
14           BasePath => '/tmp/cache',
15         } );
16
17         my $data = get( 'http://www.sn.no' );
18

DESCRIPTION

20       An implementation of http get that keeps a local cache of fetched pages
21       to avoid fetching the same data from the server if it hasn't been
22       updated. The cache is stored on disk and is thus persistent between
23       invocations.
24
25       Uses the http-headers If-Modified-Since and ETag to let the server
26       decide if the version in the cache is up-to-date or not.
27
28       The cache is implemented by modifying the LWP::UserAgent class to
29       seamlessly cache the result of all requests that can be cached.
30

INITIALIZING THE CACHE

32       HTTP::Cache::Transparent provides an init-method that sets the
33       parameters for the cache and overloads a method in LWP::UserAgent to
34       activate the cache.After init has been called, the normal LWP-methods
35       (LWP::Simple as well as the more full-fledged LWP::Request methods)
36       should be used as usual.
37
38       init
39           Initialize the HTTP cache. Takes a single parameter which is a
40           hashref containing named arguments to the object.
41
42             HTTP::Cache::Transparent::init( {
43
44               # Directory to store the cache in.
45               BasePath  => "/tmp/cache",
46
47               # How many hours should items be kept in the cache
48               # after they were last requested?
49               # Default is 8*24.
50               MaxAge    => 8*24,
51
52               # Print progress-messages to STDERR.
53               # Default is 0.
54               Verbose   => 1,
55
56               # If a request is made for a url that has been requested
57               # from the server less than NoUpdate seconds ago, the
58               # response will be generated from the cache without
59               # contacting the server.
60               # Default is 0.
61               NoUpdate  => 15*60,
62
63               # When a url has been downloaded and the response indicates that
64               # has been modified compared to the content in the cache,
65               # the ApproveContent callback is called with the HTTP::Response.
66               # The callback shall return true if the response shall be used and
67               # stored in the cache or false if the response shall be discarded
68               # and the response in the cache used instead.
69               # This mechanism can be used to work around servers that return errors
70               # intermittently. The default is to accept all responses.
71               ApproveContent => sub { return $_[0]->is_success },
72            } );
73
74           The directory where the cache is stored must be writable. It must
75           also only contain files created by HTTP::Cache::Transparent.
76
77       Initializing from use-line
78           An alternative way of initializing HTTP::Cache::Transparent is to
79           supply parameters in the use-line. This allows you to write
80
81             use HTTP::Cache::Transparent ( BasePath => '/tmp/cache' );
82
83           which is exactly equivalent to
84
85             use HTTP::Cache::Transparent;
86             HTTP::Cache::Transparent::init( BasePath => '/tmp/cache' );
87
88           The advantage to using this method is that you can do
89
90             perl -MHTTP::Cache::Transparent=BasePath,/tmp/cache myscript.pl
91
92           or even set the environment variable PERL5OPT
93
94             PERL5OPT=-MHTTP::Cache::Transparent=BasePath,/tmp/cache
95             myscript.pl
96
97           and have all the http-requests performed by myscript.pl go through
98           the cache without changing myscript.pl
99

INSPECTING CACHE BEHAVIOR

101       The HTTP::Cache::Transparent inserts three special headers in the
102       HTTP::Response object. These can be accessed via the
103       HTTP::Response::header()-method.
104
105       X-Cached
106           This header is inserted and set to 1 if the response is delivered
107           from the cache instead of from the server.
108
109       X-Content-Unchanged
110           This header is inserted and set to 1 if the content returned is the
111           same as the content returned the last time this url was fetched.
112           This header is always inserted and set to 1 when the response is
113           delivered from the cache.
114
115       X-No-Server-Contact
116           This header is inserted and set to 1 if the content returned has
117           been delivered without any contact with the external server, i.e.
118           no conditional or unconditional HTTP GET request has been sent, the
119           content has been delivered directly from cache. This may be useful
120           when seeking to control loading of the external server.
121

LIMITATIONS

123       This module has a number of limitations that you should be aware of
124       before using it.
125
126       -   There is no upper limit to how much diskspace the cache requires.
127           The only limiting mechanism is that data for urls that haven't been
128           requested in the last MaxAge hours will be removed from the cache
129           the next time the program exits.
130
131       -   Currently, only get-requests that store the result in memory (i.e.
132           do not use the option to have the result stored directly in a file
133           or delivered via a callback) is cached. I intend to remove this
134           limitation in a future version.
135
136       -   The support for Ranges is a bit primitive. It creates a new object
137           in the cache for each unique combination of url and range. This
138           will work ok as long as you always request the same range(s) for a
139           url.
140
141       -   The cache doesn't properly check and store all headers in the HTTP
142           request and response. Therefore, if you request the same url
143           repeatedly with different sets of headers (cookies, accept-encoding
144           etc), and these headers affect the response from the server, the
145           cache may return the wrong response.
146
147       -   HTTP::Cache::Transparent has not been tested with threads, and will
148           most likely not work if you use them.
149

CACHE FORMAT

151       The cache is stored on disk as one file per cached object. The filename
152       is equal to the md5sum of the url and the Range-header if it exists.
153       The file contains a set of key/value-pairs with metadata (one entry per
154       line) followed by a blank line and then the actual data returned by the
155       server.
156
157       The last modified date of the cache file is set to the time when the
158       cache object was last requested by a user.
159

AUTHOR

161       Mattias Holmlund, <$firstname -at- $lastname -dot- se>
162       <http://www.holmlund.se/mattias/>
163

GIT REPOSITORY

165       A git repository containing the source for this module can be found via
166       http://git.holmlund.se/
167
169       Copyright (C) 2004-2007 by Mattias Holmlund
170
171       This library is free software; you can redistribute it and/or modify it
172       under the same terms as Perl itself, either Perl version 5.8.4 or, at
173       your option, any later version of Perl 5 you may have available.
174
175
176
177perl v5.34.0                      2022-01-21       HTTP::Cache::Transparent(3)
Impressum