1HTTP::Proxy::BodyFilterU:s:esravCeo(n3t)ributed Perl DocHuTmTePn:t:aPtriooxny::BodyFilter::save(3)
2
6 HTTP::Proxy::BodyFilter::save - A filter that saves transferred data to
7 a file
8
10 use HTTP::Proxy;
11 use HTTP::Proxy::BodyFilter::save;
12 my $proxy = HTTP::Proxy->new;
14 # save RFC files as we browse them
16 $proxy->push_filter(
17 path => qr!/rfc\d+.txt!,
18 mime => 'text/plain',
19 response => HTTP::Proxy::BodyFilter::save->new(
20 template => '%f',
21 prefix => 'rfc',
22 keep_old => 1,
23 )
24 );
25 $proxy->start;
27
29 The HTTP::Proxy::BodyFilter::save filter can save HTTP messages
30 (responses or request) bodies to files. The name of the file is
31 determined by a template and the URI of the request.
32 Simply insert this filter in a filter stack, and it will save the data
34 as it flows through the proxy. Depending on where the filter is located
35 in the stack, the saved data can be more or less modified.
36 This filter will create directories if it needs to!
38 Note: Remember that the default "mime" parameter for "push_filter()" is
40 "text/*" and that you may need to change it for other MIME types.
41 Constructor
43 The constructor accepts quite a few options. Most of them control the
44 construction of the filename that will be used to save the response
45 body. There are two options to compute this filename:
46 · use a template
48 · use your own filename creation routine
50 The template option uses the following options:
52 template => string
54 The file name is build from the "template" option. The following
55 placeholders are available:
56 %% a percent sign
58 %h the host
59 %p the path (no leading separator)
60 %d the path (filename removed)
61 %f the filename (or 'index.html' if absent)
62 %q the query string
63 %P the path and the query string,
64 separated by '?' (if the query string is not empty)
65 "/" in the URI path are replaced by the separator used by
67 File::Spec.
68 The result of the template is modified by the no_host, no_dirs and
70 cut_dirs.
71 The default template is the local equivalent of the "%h/%P" Unix
73 path.
74 no_host => boolean
76 The "no_host" option makes %h empty. Default is false.
77 no_dirs => boolean
79 The "no_dirs" option removes all directories from %p, %P and %d.
80 Default is false.
81 cut_dirs => number
83 The "cut_dirs" options removes the first n directories from the
84 content of %p, %P and %d. Default is 0.
85 prefix => string
87 The prefix option prepends the given prefix to the filename created
88 from the template. Default is "".
89 Using your own subroutine is also possible, with the following
91 parameter:
92 filename => coderef
94 When the "filename" option is used, the "template" option and the
95 other template-related options ("no_host", "no_dirs", "cut_dirs"
96 and "prefix") are ignored.
97 The "filename" option expects a reference to a subroutine. The
99 subroutine will receive the HTTP::Message object and must return a
100 string which is the path of the file to be created (an absolute
101 path is recommended, but a relative path is accepted).
102 Returning "" or "undef" will prevent the creation of the file.
104 This lets a filter decide even more precisely what to save or not,
105 even though this should be done in the match subroutine (see
106 HTTP::Proxy's "push_filter()" method).
107 Other options help the filter decide where and when to save:
109 multiple => boolean
111 With the multiple option, saving the same file in the same
112 directory will result in the original copy of file being preserved
113 and the second copy being named file.1. If that a file is saved yet
114 again with the same name, the third copy will be named file.2, and
115 so on.
116 Default is true.
118 If multiple is set to false then a file will be overwritten by the
120 next one with the same name.
121 timestamp => boolean
123 With the "timestamp" option, the decision as to whether or not to
124 save a newer copy of a file depends on the local and remote
125 timestamp and size of the file.
126 The file is saved only if the date given in the "Last-Modified" is
128 more recent than the local file's timestamp.
129 Default is false.
131 This option is not implemented.
133 keep_old => boolean
135 The "keep_old" option will prevent the file to be saved if a file
136 with the same name already exists. Default is false.
137 No matter if multiple is set or not, the file will not be saved if
139 keep_old is set to true.
140 status => \@codes
142 The "status" option limits the status codes for which a response
143 body will be saved. The default is "[ 200 ]", which prevent saving
144 error pages (for 404 codes).
145 Examples
147 Given a request for the <http://search.cpan.org/dist/HTTP-Proxy/> URI,
148 the filename is computed as follows, depending on the constructor
149 options:
150 No options -> search.cpan.org/dist/HTTP-Proxy/index.html
152 no_host => 1 -> dist/HTTP-Proxy/index.html
154 no_dirs => 1 -> search.cpan.org/index.html
156 no_host => 1,
158 no_dirs => 1,
159 prefix => 'data' -> data/index.html
160 cut_dirs => 1 -> search.cpan.org/HTTP-Proxy/index.html
162 cut_dirs => 2 -> search.cpan.org/index.html
164
166 This filter implements several methods, which are all called
167 automatically:
168 init()
170 Handle all the parameters passed to the constructor to define the
171 filter behaviour.
172 begin()
174 Open the file to which the data will be saved.
175 filter()
177 Save all the data that goes through to the opened file.
178 end()
180 Close the file when the whole message body has been processed.
181 will_modify()
183 This method returns a false value, thus indicating to the system
184 that it will not modify data passing through.
185
187 HTTP::Proxy, HTTP::Proxy::BodyFilter.
188
190 Philippe "BooK" Bruhat, <book@cpan.org>.
191
193 Thanks to Mat Proud for asking how to store all pages which go through
194 the proxy to disk, without any processing. The further discussion we
195 had led to the writing of this class.
196 Wget(1) provided the inspiration for many of the file naming options.
198 Thanks to Nicolas Chuche for telling me about "O_EXCL".
200 Thanks to Rafaël Garcia-Suarez and David Rigaudiere for their help on
202 irc while coding the nasty "begin()" method. ";-)"
203 Thanks to Howard Jones for the inspiration and initial patch for the
205 "filename" option. Lucas Gonze provided a patch to make "status"
206 actually work.
207 Thanks to Max Maischein for detecting a bug in the parameter validation
209 for "filename" (<http://rt.cpan.org/Ticket/Display.html?id=14548>).
210 Thanks to Mark Tilford, who found out that the "filename" option was
212 incorrectly used internally
213 (<http://rt.cpan.org/Ticket/Display.html?id=18644>).
214 Thanks to Roland Stigge and Gunnar Wolf for reporting and forwarding
216 Debian bug #433951 to CPAN RT
217 (<http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=433951>,
218 <http://rt.cpan.org/Ticket/Display.html?id=33018>).
219
221 Copyright 2004-2015, Philippe Bruhat.
222
224 This module is free software; you can redistribute it or modify it
225 under the same terms as Perl itself.
226perl v5.32.0 2020-07-28 HTTP::Proxy::BodyFilter::save(3)