1MogileFS::Client(3) User Contributed Perl Documentation MogileFS::Client(3)
2
6 MogileFS::Client - Client library for the MogileFS distributed file
7 system.
8
10 use MogileFS::Client;
11 # create client object w/ server-configured namespace
13 # and IPs of trackers
14 $mogc = MogileFS::Client->new(domain => "foo.com::my_namespace",
15 hosts => ['10.0.0.2:7001', '10.0.0.3:7001']);
16 # create a file
18 # mogile is a flat namespace. no paths.
19 $key = "image_of_userid:$userid";
20 # must be configured on server
21 $class = "user_images";
22 $fh = $mogc->new_file($key, $class);
23 print $fh $data;
25 unless ($fh->close) {
27 die "Error writing file: " . $mogc->errcode . ": " . $mogc->errstr;
28 }
29 # Find the URLs that the file was replicated to.
31 # May change over time.
32 @urls = $mogc->get_paths($key);
33 # no longer want it?
35 $mogc->delete($key);
36
38 This module is a client library for the MogileFS distributed file
39 system. The class method 'new' creates a client object against a
40 particular mogilefs tracker and domain. This object may then be used to
41 store and retrieve content easily from MogileFS.
42
44 new
45 $client = MogileFS::Client->new( %OPTIONS );
46 Creates a new MogileFS::Client object.
48 Returns MogileFS::Client object on success, or dies on failure.
50 OPTIONS:
52 hosts
54 Arrayref of 'host:port' strings to connect to as backend trackers
55 in this client.
56 domain
58 String representing the mogile domain which this MogileFS client is
59 associated with. (All create/delete/fetch operations will be
60 performed against this mogile domain). See the mogadm shell command
61 and its 'domain' category of operations for information on
62 manipulating the list of possible domains on a MogileFS system.
63 reload
65 $mogc->reload( %OPTIONS )
66 Re-init the object, like you'd just reconstructed it with 'new', but
68 change it in-place instead. Useful if you have a system which reloads
69 a config file, and you want to update a singleton $mogc handle's config
70 value.
71 last_tracker
73 Returns a scalar of form "ip:port", representing the last mogilefsd
74 'tracker' server which was talked to.
75 errstr
77 Returns string representation of the last error that occurred. It
78 includes the error code (same as method 'errcode') and a space before
79 the optional English error message.
80 This isn't necessarily guaranteed to reset after a successful
82 operation. Only call it after another operation returns an error.
83 errcode
85 Returns an error code. Not a number, but a string identifier (e.g.
86 "no_domain") which is stable for use in error handling logic.
87 This isn't necessarily guaranteed to reset after a successful
89 operation. Only call it after another operation returns an error.
90 readonly
92 $is_readonly = $mogc->readonly
93 $mogc->readonly(1)
94 Getter/setter to mark this client object as read-only. Purely a local
96 operation/restriction, doesn't do a network operation to the mogilefsd
97 server.
98 new_file
100 $mogc->new_file($key)
101 $mogc->new_file($key, $class)
102 $mogc->new_file($key, $class, $content_length)
103 $mogc->new_file($key, $class, $content_length , $opts_hashref)
104 Start creating a new filehandle with the given key, and option given
106 class and options.
107 Returns a filehandle you should then print to, and later close to
109 complete the operation. NOTE: check the return value from close! If
110 your close didn't succeed, the file didn't get saved!
111 $opts_hashref can contain keys:
113 fid Explicitly specify the fid number to use, rather than it being
115 automatically allocated.
116 create_open_args
118 Hashref of extra key/value pairs to send to mogilefsd in
119 create_open phase.
120 create_close_args
122 Hashref of extra key/value pairs to send to mogilefsd in
123 create_close phase.
124 largefile
126 Use MogileFS::ClientHTTPFile which will not load the entire file
127 into memory like the default MogileFS::NewHTTPFile but requires
128 that the storage node HTTP servers support the Content-Range header
129 in PUT requests and is a little slower.
130 edit_file
132 $mogc->edit_file($key, $opts_hashref)
133 Edit the file with the the given key.
135 NOTE: edit_file is currently EXPERIMENTAL and not recommended for
137 production use. MogileFS is primarily designed for storing files for
138 later retrieval, rather than editing. Use of this function may lead to
139 poor performance and, until it has been proven mature, should be
140 considered to also potentially cause data loss.
141 NOTE: use of this function requires support for the DAV 'MOVE' verb and
143 partial PUT (i.e. Content-Range in PUT) on the back-end storage servers
144 (e.g. apache with mod_dav).
145 Returns a seekable filehandle you can read/write to. Calling this
147 function may invalidate some or all URLs you currently have for this
148 key, so you should call ->get_paths again afterwards if you need them.
149 On close of the filehandle, the new file contents will replace the
151 previous contents (and again invalidate any existing URLs).
152 By default, the file contents are preserved on open, but you may
154 specify the overwrite option to zero the file first. The seek position
155 is at the beginning of the file, but you may seek to the end to append.
156 $opts_hashref can contain keys:
158 overwrite
160 The edit will overwrite the file, equivalent to opening with '>'.
161 Default: false.
162 read_file
164 $mogc->read_file($key)
165 Read the file with the the given key.
167 Returns a seekable filehandle you can read() from. Note that you cannot
169 read line by line using <$fh> notation.
170 Takes the same options as get_paths (which is called internally to get
172 the URIs to read from).
173 store_file
175 $mogc->store_file($key, $class, $fh_or_filename[, $opts_hashref])
176 Wrapper around new_file, print, and close.
178 Given a key, class, and a filehandle or filename, stores the file
180 contents in MogileFS. Returns the number of bytes stored on success,
181 undef on failure.
182 $opts_hashref can contain keys for new_file, and also the following:
184 chunk_size
186 Number of bytes to read and write and write at once out of the
187 larger file. Defaults to 8192 bytes. Increasing this can increase
188 performance at the cost of more memory used while uploading the
189 file. Note that this mostly helps when using largefile => 1
190 store_content
192 $mogc->store_content($key, $class, $content[, $opts]);
193 Wrapper around new_file, print, and close. Given a key, class, and
195 file contents (scalar or scalarref), stores the file contents in
196 MogileFS. Returns the number of bytes stored on success, undef on
197 failure.
198 get_paths
200 @paths = $mogc->get_paths($key)
201 @paths = $mogc->get_paths($key, $no_verify_bool); # old way
202 @paths = $mogc->get_paths($key, { noverify => $bool }); # new way
203 Given a key, returns an array of all the locations (HTTP URLs) that the
205 file has been replicated to.
206 noverify
208 If the "no verify" option is set, the mogilefsd tracker doesn't
209 verify that the first item returned in the list is up/alive.
210 Skipping that check is faster, so use "noverify" if your
211 application can do it faster/smarter. For instance, when giving
212 Perlbal a list of URLs to reproxy to, Perlbal can intelligently
213 find one that's alive, so use noverify and get out of mod_perl or
214 whatever as soon as possible.
215 zone
217 If the zone option is set to 'alt', the mogilefsd tracker will use
218 the alternative IP for each host if available, while constructing
219 the paths.
220 pathcount
222 If the pathcount option is set to a positive integer greater than
223 2, the mogilefsd tracker will attempt to return that many different
224 paths (if available) to the same file. If not present or out of
225 range, this value defaults to 2.
226 get_file_data
228 $dataref = $mogc->get_file_data($key)
229 Wrapper around get_paths & LWP, which returns scalarref of file
231 contents in a scalarref.
232 Don't use for large data, as it all comes back to you in one string.
234 delete
236 $mogc->delete($key);
237 Delete a key from MogileFS.
239 rename
241 $mogc->rename($oldkey, $newkey);
242 Rename file (key) in MogileFS from oldkey to newkey. Returns true on
244 success, failure otherwise.
245 list_keys
247 $keys = $mogc->list_keys($prefix, $after[, $limit]);
248 ($after, $keys) = $mogc->list_keys($prefix, $after[, $limit]);
249 Used to get a list of keys matching a certain prefix.
251 $prefix specifies what you want to get a list of. $after is the item
253 specified as a return value from this function last time you called it.
254 $limit is optional and defaults to 1000 keys returned.
255 In list context, returns ($after, $keys). In scalar context, returns
257 arrayref of keys. The value $after is to be used as $after when you
258 call this function again.
259 When there are no more keys in the list, you will get back undef or an
261 empty list.
262 foreach_key
264 $mogc->foreach_key( %OPTIONS, sub { my $key = shift; ... } );
265 $mogc->foreach_key( prefix => "foo:", sub { my $key = shift; ... } );
266 Functional interface/wrapper around list_keys.
268 Given some %OPTIONS (currently only one, "prefix"), calls your callback
270 for each key matching the provided prefix.
271 set_pref_ip
273 $mogc->set_pref_ip({ "10.0.0.2" => "10.2.0.2" });
274 Weird option for old, weird network architecture. Sets a mapping table
276 of preferred alternate IPs, if reachable. For instance, if trying to
277 connect to 10.0.0.2 in the above example, the module would instead try
278 to connect to 10.2.0.2 quickly first, then then fall back to 10.0.0.2
279 if 10.2.0.2 wasn't reachable.
280
282 WRITEME
283
285 add_hook
286 WRITEME
287 add_backend_hook
289 WRITEME
290
292 <http://www.danga.com/mogilefs/>
293
295 This module is Copyright 2003-2004 Brad Fitzpatrick, and copyright
296 2005-2007 Six Apart, Ltd.
297 All rights reserved.
299 You may distribute under the terms of either the GNU General Public
301 License or the Artistic License, as specified in the Perl README file.
302
304 This is free software. IT COMES WITHOUT WARRANTY OF ANY KIND.
305
307 Brad Fitzpatrick <brad@danga.com>
308 Brad Whitaker <whitaker@danga.com>
310 Mark Smith <marksmith@danga.com>
312perl v5.12.1 2010-04-02 MogileFS::Client(3)