1HTTP::Proxy::BodyFilterU:s:esravCeo(n3t)ributed Perl DocHuTmTePn:t:aPtriooxny::BodyFilter::save(3)
2
6 HTTP::Proxy::BodyFilter::save - A filter that saves transfered data to
7 a file
8
10 use HTTP::Proxy;
11 use HTTP::Proxy::BodyFilter::save;
12 my $proxy = HTTP::Proxy->new;
14 # save RFC files as we browse them
16 $proxy->push_filter(
17 path => qr!/rfc\d+.txt!,
18 mime => 'text/plain',
19 response => HTTP::Proxy::BodyFilter::save->new(
20 template => '%f',
21 prefix => 'rfc',
22 keep_old => 1,
23 );
24 );
25 $proxy->start;
27
29 The HTTP::Proxy::BodyFilter::save filter can save HTTP messages
30 (responses or request) bodies to files. The name of the file is deter‐
31 mined by a template and the URI of the request.
32 Simply insert this filter in a filter stack, and it will save the data
34 as it flows through the proxy. Depending on where the filter is located
35 in the stack, the saved data can be more or less modified.
36 This filter will create directories if it needs to!
38 Note: Remember that the default "mime" parameter for "push_filter()" is
40 "text/*" and that you may need to change it for other MIME types.
41 Constructor
43 The constructor accepts quite a few options. Most of them control the
45 construction of the filename that will be used to save the response
46 body. There are two options to compute this filename:
47 · use a template
49 · use your own filename creation routine
51 The template option uses the following options:
53 template => string
55 The file name is build from the "template" option. The following
56 placeholders are available:
57 %% a percent sign
59 %h the host
60 %p the path (no leading separator)
61 %d the path (filename removed)
62 %f the filename (or 'index.html' if absent)
63 %q the query string
64 %P the path and the query string,
65 separated by '?' (if the query string is not empty)
66 "/" in the URI path are replaced by the separator used by
68 File::Spec.
69 The result of the template is modified by the no_host, no_dirs and
71 cut_dirs.
72 The default template is the local equivalent of the "%h/%P" Unix
74 path.
75 no_host => boolean
77 The "no_host" option makes %h empty. Default is false.
78 no_dirs => boolean
80 The "no_dirs" option removes all directories from %p, %P and %d.
81 Default is false.
82 cut_dirs => number
84 The "cut_dirs" options removes the first n directories from the
85 content of %p, %P and %d. Default is 0.
86 prefix => string
88 The prefix option prepends the given prefix to the filename created
89 from the template. Default is "".
90 Using your own subroutine is also possible, with the following parame‐
92 ter:
93 filename => coderef
95 When the "filename" option is used, the "template" option and the
96 other template-related options ("no_host", "no_dirs", "cut_dirs"
97 and "prefix") are ignored.
98 The "filename" option expects a reference to a subroutine. The sub‐
100 routine will receive the HTTP::Message object and must return a
101 string which is the path of the file to be created (an absolute
102 path is recommended, but a relative path is accepted).
103 Returning "" or "undef" will prevent the creation of the file.
105 This lets a filter decide even more precisely what to save or not,
106 even though this should be done in the match subroutine (see
107 HTTP::Proxy's "push_filter()" method).
108 Other options help the filter decide where and when to save:
110 multiple => boolean
112 With the multiple option, saving the same file in the same direc‐
113 tory will result in the original copy of file being preserved and
114 the second copy being named file.1. If that a file is saved yet
115 again with the same name, the third copy will be named file.2, and
116 so on.
117 Default is true.
119 If multiple is set to false then a file will be overwritten by the
121 next one with the same name.
122 timestamp => boolean
124 With the "timestamp" option, the decision as to whether or not to
125 save a newer copy of a file depends on the local and remote time‐
126 stamp and size of the file.
127 The file is saved only if the date given in the "Last-Modified" is
129 more recent than the local file's timestamp.
130 Default is false.
132 keep_old => boolean
134 The "keep_old" option will prevent the file to be saved if a file
135 with the same name already exists. Default is false.
136 No matter if multiple is set or not, the file will not be saved if
138 keep_old is set to true.
139 status => \@codes
141 The "status" option limits the status codes for which a response
142 body will be saved. The default is "[ 200 ]", which prevent saving
143 error pages (for 404 codes).
144 Examples
146 Given a request for the <http://search.cpan.org/dist/HTTP-Proxy/> URI,
148 the filename is computed as follows, depending on the constructor
149 options:
150 No options -> search.cpan.org/dist/HTTP-Proxy/index.html
152 no_host => 1 -> dist/HTTP-Proxy/index.html
154 no_dirs => 1 -> search.cpan.org/index.html
156 no_host => 1,
158 no_dirs => 1,
159 prefix => 'data' -> data/index.html
160 cut_dirs => 1 -> search.cpan.org/HTTP-Proxy/index.html
162 cut_dirs => 2 -> search.cpan.org/index.html
164
166 This filter implements several methods, which are all called atuomati‐
167 cally:
168 init()
170 Handle all the parameters passed to the constructor to define the
171 filter behaviour.
172 begin()
174 Open the file to which the data will be saved.
175 filter()
177 Save all the data that goes through to the opened file.
178 end()
180 Close the file when the whole message body has been processed.
181 will_modify()
183 This method returns a false value, thus indicating to the system
184 that it will not modify data passing through.
185
187 HTTP::Proxy, HTTP::Proxy::BodyFilter.
188
190 Philippe "BooK" Bruhat, <book@cpan.org>.
191
193 Thanks to Mat Proud for asking how to store all pages which go through
194 the proxy to disk, without any processing. The further discussion we
195 had led to the writing of this class.
196 Wget(1) provided the inspiration for many of the file naming options.
198 Thanks to Nicolas Chuche for telling me about "O_EXCL".
200 Thanks to Rafaël Garcia-Suarez and David Rigaudiere for their help on
202 irc while coding the nasty "begin()" method. ";-)"
203 Thanks to Howard Jones for the inspiration and initial patch for the
205 "filename" option. Lucas Gonze provided a patch to make "status" actu‐
206 ally work.
207 Thanks to Max Maischein for detecting a bug in the paramerter valida‐
209 tion for "filename".
210
212 Copyright 2004-2006, Philippe Bruhat.
213
215 This module is free software; you can redistribute it or modify it
216 under the same terms as Perl itself.
217perl v5.8.8 2006-09-04 HTTP::Proxy::BodyFilter::save(3)