1File::Fetch(3)        User Contributed Perl Documentation       File::Fetch(3)
2
3
4

NAME

6       File::Fetch - A generic file fetching mechanism
7

SYNOPSIS

9           use File::Fetch;
10
11           ### build a File::Fetch object ###
12           my $ff = File::Fetch->new(uri => 'http://some.where.com/dir/a.txt');
13
14           ### fetch the uri to cwd() ###
15           my $where = $ff->fetch() or die $ff->error;
16
17           ### fetch the uri to /tmp ###
18           my $where = $ff->fetch( to => '/tmp' );
19
20           ### parsed bits from the uri ###
21           $ff->uri;
22           $ff->scheme;
23           $ff->host;
24           $ff->path;
25           $ff->file;
26

DESCRIPTION

28       File::Fetch is a generic file fetching mechanism.
29
30       It allows you to fetch any file pointed to by a "ftp", "http", "file",
31       or "rsync" uri by a number of different means.
32
33       See the "HOW IT WORKS" section further down for details.
34

ACCESSORS

36       A "File::Fetch" object has the following accessors
37
38       $ff->uri
39           The uri you passed to the constructor
40
41       $ff->scheme
42           The scheme from the uri (like 'file', 'http', etc)
43
44       $ff->host
45           The hostname in the uri, will be empty for a 'file' scheme.
46
47       $ff->path
48           The path from the uri, will be at least a single '/'.
49
50       $ff->file
51           The name of the remote file. For the local file name, the result of
52           $ff->output_file will be used.
53
54       $ff->output_file
55           The name of the output file. This is the same as $ff->file, but any
56           query parameters are stripped off. For example:
57
58               http://example.com/index.html?x=y
59
60           would make the output file be "index.html" rather than
61           "index.html?x=y".
62

METHODS

64       $ff = File::Fetch->new( uri => 'http://some.where.com/dir/file.txt' );
65
66       Parses the uri and creates a corresponding File::Fetch::Item object,
67       that is ready to be "fetch"ed and returns it.
68
69       Returns false on failure.
70
71       $ff->fetch( [to => /my/output/dir/] )
72
73       Fetches the file you requested. By default it writes to "cwd()", but
74       you can override that by specifying the "to" argument.
75
76       Returns the full path to the downloaded file on success, and false on
77       failure.
78
79       $ff->error([BOOL])
80
81       Returns the last encountered error as string.  Pass it a true value to
82       get the "Carp::longmess()" output instead.
83

HOW IT WORKS

85       File::Fetch is able to fetch a variety of uris, by using several exter‐
86       nal programs and modules.
87
88       Below is a mapping of what utilities will be used in what order for
89       what schemes, if available:
90
91           file    => LWP, file
92           http    => LWP, wget, curl, lynx
93           ftp     => LWP, Net::FTP, wget, curl, ncftp, ftp
94           rsync   => rsync
95
96       If you'd like to disable the use of one or more of these utilities
97       and/or modules, see the $BLACKLIST variable further down.
98
99       If a utility or module isn't available, it will be marked in a cache
100       (see the $METHOD_FAIL variable further down), so it will not be tried
101       again. The "fetch" method will only fail when all options are
102       exhausted, and it was not able to retrieve the file.
103
104       A special note about fetching files from an ftp uri:
105
106       By default, all ftp connections are done in passive mode. To change
107       that, see the $FTP_PASSIVE variable further down.
108
109       Furthermore, ftp uris only support anonymous connections, so no named
110       user/password pair can be passed along.
111
112       "/bin/ftp" is blacklisted by default; see the $BLACKLIST variable fur‐
113       ther down.
114

GLOBAL VARIABLES

116       The behaviour of File::Fetch can be altered by changing the following
117       global variables:
118
119       $File::Fetch::FROM_EMAIL
120
121       This is the email address that will be sent as your anonymous ftp pass‐
122       word.
123
124       Default is "File-Fetch@example.com".
125
126       $File::Fetch::USER_AGENT
127
128       This is the useragent as "LWP" will report it.
129
130       Default is "File::Fetch/$VERSION".
131
132       $File::Fetch::FTP_PASSIVE
133
134       This variable controls whether the environment variable "FTP_PASSIVE"
135       and any passive switches to commandline tools will be set to true.
136
137       Default value is 1.
138
139       Note: When $FTP_PASSIVE is true, "ncftp" will not be used to fetch
140       files, since passive mode can only be set interactively for this binary
141
142       $File::Fetch::TIMEOUT
143
144       When set, controls the network timeout (counted in seconds).
145
146       Default value is 0.
147
148       $File::Fetch::WARN
149
150       This variable controls whether errors encountered internally by
151       "File::Fetch" should be "carp"'d or not.
152
153       Set to false to silence warnings. Inspect the output of the "error()"
154       method manually to see what went wrong.
155
156       Defaults to "true".
157
158       $File::Fetch::DEBUG
159
160       This enables debugging output when calling commandline utilities to
161       fetch files.  This also enables "Carp::longmess" errors, instead of the
162       regular "carp" errors.
163
164       Good for tracking down why things don't work with your particular set‐
165       up.
166
167       Default is 0.
168
169       $File::Fetch::BLACKLIST
170
171       This is an array ref holding blacklisted modules/utilities for fetching
172       files with.
173
174       To disallow the use of, for example, "LWP" and "Net::FTP", you could
175       set $File::Fetch::BLACKLIST to:
176
177           $File::Fetch::BLACKLIST = [qw⎪lwp netftp⎪]
178
179       The default blacklist is [qw⎪ftp⎪], as "/bin/ftp" is rather unreliable.
180
181       See the note on "MAPPING" below.
182
183       $File::Fetch::METHOD_FAIL
184
185       This is a hashref registering what modules/utilities were known to fail
186       for fetching files (mostly because they weren't installed).
187
188       You can reset this cache by assigning an empty hashref to it, or indi‐
189       vidually remove keys.
190
191       See the note on "MAPPING" below.
192

MAPPING

194       Here's a quick mapping for the utilities/modules, and their names for
195       the $BLACKLIST, $METHOD_FAIL and other internal functions.
196
197           LWP         => lwp
198           Net::FTP    => netftp
199           wget        => wget
200           lynx        => lynx
201           ncftp       => ncftp
202           ftp         => ftp
203           curl        => curl
204           rsync       => rsync
205

FREQUENTLY ASKED QUESTIONS

207       So how do I use a proxy with File::Fetch?
208
209       "File::Fetch" currently only supports proxies with LWP::UserAgent.  You
210       will need to set your environment variables accordingly. For example,
211       to use an ftp proxy:
212
213           $ENV{ftp_proxy} = 'foo.com';
214
215       Refer to the LWP::UserAgent manpage for more details.
216
217       I used 'lynx' to fetch a file, but its contents is all wrong!
218
219       "lynx" can only fetch remote files by dumping its contents to "STDOUT",
220       which we in turn capture. If that content is a 'custom' error file
221       (like, say, a "404 handler"), you will get that contents instead.
222
223       Sadly, "lynx" doesn't support any options to return a different exit
224       code on non-"200 OK" status, giving us no way to tell the difference
225       between a 'successfull' fetch and a custom error page.
226
227       Therefor, we recommend to only use "lynx" as a last resort. This is why
228       it is at the back of our list of methods to try as well.
229
230       Files I'm trying to fetch have reserved characters or non-ASCII charac‐
231       ters in them. What do I do?
232
233       "File::Fetch" is relatively smart about things. When trying to write a
234       file to disk, it removes the "query parameters" (see the "output_file"
235       method for details) from the file name before creating it. In most
236       cases this suffices.
237
238       If you have any other characters you need to escape, please install the
239       "URI::Escape" module from CPAN, and pre-encode your URI before passing
240       it to "File::Fetch". You can read about the details of URIs and URI
241       encoding here:
242
243         http://www.faqs.org/rfcs/rfc2396.html
244

TODO

246       Implement $PREFER_BIN
247           To indicate to rather use commandline tools than modules
248

AUTHORS

250       This module by Jos Boumans <kane@cpan.org>.
251

COPYRIGHT

253       This module is copyright (c) 2003-2007 Jos Boumans <kane@cpan.org>. All
254       rights reserved.
255
256       This library is free software; you may redistribute and/or modify it
257       under the same terms as Perl itself.
258
259
260
261perl v5.8.8                       2007-01-26                    File::Fetch(3)