1MogileFS::Client(3) User Contributed Perl Documentation MogileFS::Client(3)
2
6 MogileFS::Client - Client library for the MogileFS distributed file
7 system.
8
10 use MogileFS::Client;
11 # create client object w/ server-configured namespace
13 # and IPs of trackers
14 $mogc = MogileFS::Client->new(domain => "foo.com::my_namespace",
15 hosts => ['10.0.0.2:7001', '10.0.0.3:7001']);
16 # create a file
18 # mogile is a flat namespace. no paths.
19 $key = "image_of_userid:$userid";
20 # must be configured on server
21 $class = "user_images";
22 $fh = $mogc->new_file($key, $class);
23 print $fh $data;
25 unless ($fh->close) {
27 die "Error writing file: " . $mogc->errcode . ": " . $mogc->errstr;
28 }
29 # Find the URLs that the file was replicated to.
31 # May change over time.
32 @urls = $mogc->get_paths($key);
33 # no longer want it?
35 $mogc->delete($key);
36
38 This module is a client library for the MogileFS distributed file
39 system. The class method 'new' creates a client object against a
40 particular mogilefs tracker and domain. This object may then be used to
41 store and retrieve content easily from MogileFS.
42
44 new
45 $client = MogileFS::Client->new( %OPTIONS );
46 Creates a new MogileFS::Client object.
48 Returns MogileFS::Client object on success, or dies on failure.
50 OPTIONS:
52 hosts
54 Arrayref of 'host:port' strings to connect to as backend trackers
55 in this client.
56 domain
58 String representing the mogile domain which this MogileFS client is
59 associated with. (All create/delete/fetch operations will be
60 performed against this mogile domain). See the mogadm shell command
61 and its 'domain' category of operations for information on
62 manipulating the list of possible domains on a MogileFS system.
63 reload
65 $mogc->reload( %OPTIONS )
66 Re-init the object, like you'd just reconstructed it with 'new', but
68 change it in-place instead. Useful if you have a system which reloads
69 a config file, and you want to update a singleton $mogc handle's config
70 value.
71 last_tracker
73 Returns a scalar of form "ip:port", representing the last mogilefsd
74 'tracker' server which was talked to.
75 errstr
77 Returns string representation of the last error that occurred. It
78 includes the error code (same as method 'errcode') and a space before
79 the optional English error message.
80 This isn't necessarily guaranteed to reset after a successful
82 operation. Only call it after another operation returns an error.
83 errcode
85 Returns an error code. Not a number, but a string identifier (e.g.
86 "no_domain") which is stable for use in error handling logic.
87 This isn't necessarily guaranteed to reset after a successful
89 operation. Only call it after another operation returns an error.
90 force_disconnect
92 Forces the client to disconnect from the tracker, causing it to
93 reconnect when the next request is made. It will reconnect to a
94 different tracker if possible. A paranoid application may wish to do
95 to this before retrying a failed command, on the off chance that
96 another tracker may be working better.
97 readonly
99 $is_readonly = $mogc->readonly
100 $mogc->readonly(1)
101 Getter/setter to mark this client object as read-only. Purely a local
103 operation/restriction, doesn't do a network operation to the mogilefsd
104 server.
105 new_file
107 $mogc->new_file($key)
108 $mogc->new_file($key, $class)
109 $mogc->new_file($key, $class, $content_length)
110 $mogc->new_file($key, $class, $content_length , $opts_hashref)
111 Start creating a new filehandle with the given key, and option given
113 class and options.
114 Returns a filehandle you should then print to, and later close to
116 complete the operation. NOTE: check the return value from close! If
117 your close didn't succeed, the file didn't get saved!
118 $opts_hashref can contain keys:
120 fid Explicitly specify the fid number to use, rather than it being
122 automatically allocated.
123 create_open_args
125 Hashref of extra key/value pairs to send to mogilefsd in
126 create_open phase.
127 create_close_args
129 Hashref of extra key/value pairs to send to mogilefsd in
130 create_close phase.
131 largefile
133 Use MogileFS::ClientHTTPFile which will not load the entire file
134 into memory like the default MogileFS::NewHTTPFile but requires
135 that the storage node HTTP servers support the Content-Range header
136 in PUT requests and is a little slower.
137 edit_file
139 $mogc->edit_file($key, $opts_hashref)
140 Edit the file with the the given key.
142 NOTE: edit_file is currently EXPERIMENTAL and not recommended for
144 production use. MogileFS is primarily designed for storing files for
145 later retrieval, rather than editing. Use of this function may lead to
146 poor performance and, until it has been proven mature, should be
147 considered to also potentially cause data loss.
148 NOTE: use of this function requires support for the DAV 'MOVE' verb and
150 partial PUT (i.e. Content-Range in PUT) on the back-end storage servers
151 (e.g. apache with mod_dav).
152 Returns a seekable filehandle you can read/write to. Calling this
154 function may invalidate some or all URLs you currently have for this
155 key, so you should call ->get_paths again afterwards if you need them.
156 On close of the filehandle, the new file contents will replace the
158 previous contents (and again invalidate any existing URLs).
159 By default, the file contents are preserved on open, but you may
161 specify the overwrite option to zero the file first. The seek position
162 is at the beginning of the file, but you may seek to the end to append.
163 $opts_hashref can contain keys:
165 overwrite
167 The edit will overwrite the file, equivalent to opening with '>'.
168 Default: false.
169 read_file
171 $mogc->read_file($key)
172 Read the file with the the given key.
174 Returns a seekable filehandle you can read() from. Note that you cannot
176 read line by line using <$fh> notation.
177 Takes the same options as get_paths (which is called internally to get
179 the URIs to read from).
180 store_file
182 $mogc->store_file($key, $class, $fh_or_filename[, $opts_hashref])
183 Wrapper around new_file, print, and close.
185 Given a key, class, and a filehandle or filename, stores the file
187 contents in MogileFS. Returns the number of bytes stored on success,
188 undef on failure.
189 $opts_hashref can contain keys for new_file, and also the following:
191 chunk_size
193 Number of bytes to read and write and write at once out of the
194 larger file. Defaults to 8192 bytes. Increasing this can increase
195 performance at the cost of more memory used while uploading the
196 file. Note that this mostly helps when using largefile => 1
197 store_content
199 $mogc->store_content($key, $class, $content[, $opts]);
200 Wrapper around new_file, print, and close. Given a key, class, and
202 file contents (scalar or scalarref), stores the file contents in
203 MogileFS. Returns the number of bytes stored on success, undef on
204 failure.
205 get_paths
207 @paths = $mogc->get_paths($key)
208 @paths = $mogc->get_paths($key, $no_verify_bool); # old way
209 @paths = $mogc->get_paths($key, { noverify => $bool }); # new way
210 Given a key, returns an array of all the locations (HTTP URLs) that the
212 file has been replicated to.
213 noverify
215 If the "no verify" option is set, the mogilefsd tracker doesn't
216 verify that the first item returned in the list is up/alive.
217 Skipping that check is faster, so use "noverify" if your
218 application can do it faster/smarter. For instance, when giving
219 Perlbal a list of URLs to reproxy to, Perlbal can intelligently
220 find one that's alive, so use noverify and get out of mod_perl or
221 whatever as soon as possible.
222 zone
224 If the zone option is set to 'alt', the mogilefsd tracker will use
225 the alternative IP for each host if available, while constructing
226 the paths.
227 pathcount
229 If the pathcount option is set to a positive integer greater than
230 2, the mogilefsd tracker will attempt to return that many different
231 paths (if available) to the same file. If not present or out of
232 range, this value defaults to 2.
233 get_file_data
235 $dataref = $mogc->get_file_data($key)
236 Wrapper around get_paths & LWP, which returns scalarref of file
238 contents in a scalarref.
239 Don't use for large data, as it all comes back to you in one string.
241 delete
243 $mogc->delete($key);
244 Delete a key from MogileFS.
246 rename
248 $mogc->rename($oldkey, $newkey);
249 Rename file (key) in MogileFS from oldkey to newkey. Returns true on
251 success, failure otherwise.
252 file_debug
254 my $info_gob = $mogc->file_debug(fid => $fid);
255 ... or ...
256 my $info_gob = $mogc->file_debug(key => $key);
257 Thoroughly search for any database notes about a particular fid.
259 Searchable by raw fidid, or by domain and key. Use sparingly. Command
260 hits the master database numerous times, and if you're using it in
261 production something is likely very wrong.
262 To be used with troubleshooting broken/odd files and errors from
264 mogilefsd.
265 file_info
267 my $fid = $mogc->file_info($key, { devices => 0 });
268 Used to return metadata about a file. Returns the domain, class,
270 expected length, devcount, etc. Optionally device ids (not paths) can
271 be returned as well.
272 Should be used for informational purposes, and not usually for
274 dynamically serving files.
275 list_keys
277 $keys = $mogc->list_keys($prefix, $after[, $limit]);
278 ($after, $keys) = $mogc->list_keys($prefix, $after[, $limit]);
279 Used to get a list of keys matching a certain prefix.
281 $prefix specifies what you want to get a list of. $after is the item
283 specified as a return value from this function last time you called it.
284 $limit is optional and defaults to 1000 keys returned.
285 In list context, returns ($after, $keys). In scalar context, returns
287 arrayref of keys. The value $after is to be used as $after when you
288 call this function again.
289 When there are no more keys in the list, you will get back undef or an
291 empty list.
292 foreach_key
294 $mogc->foreach_key( %OPTIONS, sub { my $key = shift; ... } );
295 $mogc->foreach_key( prefix => "foo:", sub { my $key = shift; ... } );
296 Functional interface/wrapper around list_keys.
298 Given some %OPTIONS (currently only one, "prefix"), calls your callback
300 for each key matching the provided prefix.
301 update_class
303 $mogc->update_class($key, $newclass);
304 Update the replication class of a pre-existing file, causing the file
306 to become more or less replicated.
307 set_pref_ip
309 $mogc->set_pref_ip({ "10.0.0.2" => "10.2.0.2" });
310 Weird option for old, weird network architecture. Sets a mapping table
312 of preferred alternate IPs, if reachable. For instance, if trying to
313 connect to 10.0.0.2 in the above example, the module would instead try
314 to connect to 10.2.0.2 quickly first, then then fall back to 10.0.0.2
315 if 10.2.0.2 wasn't reachable.
316
318 WRITEME
319
321 add_hook
322 WRITEME
323 add_backend_hook
325 WRITEME
326
328 <http://www.danga.com/mogilefs/>
329
331 This module is Copyright 2003-2004 Brad Fitzpatrick, and copyright
332 2005-2007 Six Apart, Ltd.
333 All rights reserved.
335 You may distribute under the terms of either the GNU General Public
337 License or the Artistic License, as specified in the Perl README file.
338
340 This is free software. IT COMES WITHOUT WARRANTY OF ANY KIND.
341
343 Brad Fitzpatrick <brad@danga.com>
344 Brad Whitaker <whitaker@danga.com>
346 Mark Smith <marksmith@danga.com>
348perl v5.30.1 2020-01-30 MogileFS::Client(3)