1MogileFS::Client(3)   User Contributed Perl Documentation  MogileFS::Client(3)
2
3
4

NAME

6       MogileFS::Client - Client library for the MogileFS distributed file
7       system.
8

SYNOPSIS

10        use MogileFS::Client;
11
12        # create client object w/ server-configured namespace
13        # and IPs of trackers
14        $mogc = MogileFS::Client->new(domain => "foo.com::my_namespace",
15                                      hosts  => ['10.0.0.2:7001', '10.0.0.3:7001']);
16
17        # create a file
18        # mogile is a flat namespace.  no paths.
19        $key   = "image_of_userid:$userid";
20        # must be configured on server
21        $class = "user_images";
22        $fh = $mogc->new_file($key, $class);
23
24        print $fh $data;
25
26        unless ($fh->close) {
27           die "Error writing file: " . $mogc->errcode . ": " . $mogc->errstr;
28        }
29
30        # Find the URLs that the file was replicated to.
31        # May change over time.
32        @urls = $mogc->get_paths($key);
33
34        # no longer want it?
35        $mogc->delete($key);
36

DESCRIPTION

38       This module is a client library for the MogileFS distributed file
39       system. The class method 'new' creates a client object against a
40       particular mogilefs tracker and domain. This object may then be used to
41       store and retrieve content easily from MogileFS.
42

METHODS

44   new
45         $client = MogileFS::Client->new( %OPTIONS );
46
47       Creates a new MogileFS::Client object.
48
49       Returns MogileFS::Client object on success, or dies on failure.
50
51       OPTIONS:
52
53       hosts
54           Arrayref of 'host:port' strings to connect to as backend trackers
55           in this client.
56
57       domain
58           String representing the mogile domain which this MogileFS client is
59           associated with. (All create/delete/fetch operations will be
60           performed against this mogile domain). See the mogadm shell command
61           and its 'domain' category of operations for information on
62           manipulating the list of possible domains on a MogileFS system.
63
64   reload
65         $mogc->reload( %OPTIONS )
66
67       Re-init the object, like you'd just reconstructed it with 'new', but
68       change it in-place instead.  Useful if you have a system which reloads
69       a config file, and you want to update a singleton $mogc handle's config
70       value.
71
72   last_tracker
73       Returns a scalar of form "ip:port", representing the last mogilefsd
74       'tracker' server which was talked to.
75
76   errstr
77       Returns string representation of the last error that occurred.  It
78       includes the error code (same as method 'errcode') and a space before
79       the optional English error message.
80
81       This isn't necessarily guaranteed to reset after a successful
82       operation.  Only call it after another operation returns an error.
83
84   errcode
85       Returns an error code.  Not a number, but a string identifier (e.g.
86       "no_domain") which is stable for use in error handling logic.
87
88       This isn't necessarily guaranteed to reset after a successful
89       operation.  Only call it after another operation returns an error.
90
91   force_disconnect
92       Forces the client to disconnect from the tracker, causing it to
93       reconnect when the next request is made.  It will reconnect to a
94       different tracker if possible.  A paranoid application may wish to do
95       to this before retrying a failed command, on the off chance that
96       another tracker may be working better.
97
98   readonly
99         $is_readonly = $mogc->readonly
100         $mogc->readonly(1)
101
102       Getter/setter to mark this client object as read-only.  Purely a local
103       operation/restriction, doesn't do a network operation to the mogilefsd
104       server.
105
106   new_file
107         $mogc->new_file($key)
108         $mogc->new_file($key, $class)
109         $mogc->new_file($key, $class, $content_length)
110         $mogc->new_file($key, $class, $content_length , $opts_hashref)
111
112       Start creating a new filehandle with the given key, and option given
113       class and options.
114
115       Returns a filehandle you should then print to, and later close to
116       complete the operation.  NOTE: check the return value from close!  If
117       your close didn't succeed, the file didn't get saved!
118
119       $opts_hashref can contain keys:
120
121       fid Explicitly specify the fid number to use, rather than it being
122           automatically allocated.
123
124       create_open_args
125           Hashref of extra key/value pairs to send to mogilefsd in
126           create_open phase.
127
128       create_close_args
129           Hashref of extra key/value pairs to send to mogilefsd in
130           create_close phase.
131
132       largefile
133           Use MogileFS::ClientHTTPFile which will not load the entire file
134           into memory like the default MogileFS::NewHTTPFile but requires
135           that the storage node HTTP servers support the Content-Range header
136           in PUT requests and is a little slower.
137
138   edit_file
139         $mogc->edit_file($key, $opts_hashref)
140
141       Edit the file with the the given key.
142
143       NOTE: edit_file is currently EXPERIMENTAL and not recommended for
144       production use. MogileFS is primarily designed for storing files for
145       later retrieval, rather than editing.  Use of this function may lead to
146       poor performance and, until it has been proven mature, should be
147       considered to also potentially cause data loss.
148
149       NOTE: use of this function requires support for the DAV 'MOVE' verb and
150       partial PUT (i.e. Content-Range in PUT) on the back-end storage servers
151       (e.g. apache with mod_dav).
152
153       Returns a seekable filehandle you can read/write to. Calling this
154       function may invalidate some or all URLs you currently have for this
155       key, so you should call ->get_paths again afterwards if you need them.
156
157       On close of the filehandle, the new file contents will replace the
158       previous contents (and again invalidate any existing URLs).
159
160       By default, the file contents are preserved on open, but you may
161       specify the overwrite option to zero the file first. The seek position
162       is at the beginning of the file, but you may seek to the end to append.
163
164       $opts_hashref can contain keys:
165
166       overwrite
167           The edit will overwrite the file, equivalent to opening with '>'.
168           Default: false.
169
170   read_file
171         $mogc->read_file($key)
172
173       Read the file with the the given key.
174
175       Returns a seekable filehandle you can read() from. Note that you cannot
176       read line by line using <$fh> notation.
177
178       Takes the same options as get_paths (which is called internally to get
179       the URIs to read from).
180
181   store_file
182         $mogc->store_file($key, $class, $fh_or_filename[, $opts_hashref])
183
184       Wrapper around new_file, print, and close.
185
186       Given a key, class, and a filehandle or filename, stores the file
187       contents in MogileFS.  Returns the number of bytes stored on success,
188       undef on failure.
189
190       $opts_hashref can contain keys for new_file, and also the following:
191
192       chunk_size
193           Number of bytes to read and write and write at once out of the
194           larger file.  Defaults to 8192 bytes. Increasing this can increase
195           performance at the cost of more memory used while uploading the
196           file.  Note that this mostly helps when using largefile => 1
197
198   store_content
199           $mogc->store_content($key, $class, $content[, $opts]);
200
201       Wrapper around new_file, print, and close.  Given a key, class, and
202       file contents (scalar or scalarref), stores the file contents in
203       MogileFS. Returns the number of bytes stored on success, undef on
204       failure.
205
206   get_paths
207         @paths = $mogc->get_paths($key)
208         @paths = $mogc->get_paths($key, $no_verify_bool); # old way
209         @paths = $mogc->get_paths($key, { noverify => $bool }); # new way
210
211       Given a key, returns an array of all the locations (HTTP URLs) that the
212       file has been replicated to.
213
214       noverify
215           If the "no verify" option is set, the mogilefsd tracker doesn't
216           verify that the first item returned in the list is up/alive.
217           Skipping that check is faster, so use "noverify" if your
218           application can do it faster/smarter.  For instance, when giving
219           Perlbal a list of URLs to reproxy to, Perlbal can intelligently
220           find one that's alive, so use noverify and get out of mod_perl or
221           whatever as soon as possible.
222
223       zone
224           If the zone option is set to 'alt', the mogilefsd tracker will use
225           the alternative IP for each host if available, while constructing
226           the paths.
227
228       pathcount
229           If the pathcount option is set to a positive integer greater than
230           2, the mogilefsd tracker will attempt to return that many different
231           paths (if available) to the same file. If not present or out of
232           range, this value defaults to 2.
233
234   get_file_data
235         $dataref = $mogc->get_file_data($key)
236
237       Wrapper around get_paths & LWP, which returns scalarref of file
238       contents in a scalarref.
239
240       Don't use for large data, as it all comes back to you in one string.
241
242   delete
243           $mogc->delete($key);
244
245       Delete a key from MogileFS.
246
247   rename
248         $mogc->rename($oldkey, $newkey);
249
250       Rename file (key) in MogileFS from oldkey to newkey.  Returns true on
251       success, failure otherwise.
252
253   file_debug
254           my $info_gob = $mogc->file_debug(fid => $fid);
255           ... or ...
256           my $info_gob = $mogc->file_debug(key => $key);
257
258       Thoroughly search for any database notes about a particular fid.
259       Searchable by raw fidid, or by domain and key. Use sparingly. Command
260       hits the master database numerous times, and if you're using it in
261       production something is likely very wrong.
262
263       To be used with troubleshooting broken/odd files and errors from
264       mogilefsd.
265
266   file_info
267           my $fid = $mogc->file_info($key, { devices => 0 });
268
269       Used to return metadata about a file. Returns the domain, class,
270       expected length, devcount, etc. Optionally device ids (not paths) can
271       be returned as well.
272
273       Should be used for informational purposes, and not usually for
274       dynamically serving files.
275
276   list_keys
277           $keys = $mogc->list_keys($prefix, $after[, $limit]);
278           ($after, $keys) = $mogc->list_keys($prefix, $after[, $limit]);
279
280       Used to get a list of keys matching a certain prefix.
281
282       $prefix specifies what you want to get a list of.  $after is the item
283       specified as a return value from this function last time you called it.
284       $limit is optional and defaults to 1000 keys returned.
285
286       In list context, returns ($after, $keys).  In scalar context, returns
287       arrayref of keys.  The value $after is to be used as $after when you
288       call this function again.
289
290       When there are no more keys in the list, you will get back undef or an
291       empty list.
292
293   foreach_key
294         $mogc->foreach_key( %OPTIONS, sub { my $key = shift; ... } );
295         $mogc->foreach_key( prefix => "foo:", sub { my $key = shift; ... } );
296
297       Functional interface/wrapper around list_keys.
298
299       Given some %OPTIONS (currently only one, "prefix"), calls your callback
300       for each key matching the provided prefix.
301
302   update_class
303           $mogc->update_class($key, $newclass);
304
305       Update the replication class of a pre-existing file, causing the file
306       to become more or less replicated.
307
308   set_pref_ip
309         $mogc->set_pref_ip({ "10.0.0.2" => "10.2.0.2" });
310
311       Weird option for old, weird network architecture.  Sets a mapping table
312       of preferred alternate IPs, if reachable.  For instance, if trying to
313       connect to 10.0.0.2 in the above example, the module would instead try
314       to connect to 10.2.0.2 quickly first, then then fall back to 10.0.0.2
315       if 10.2.0.2 wasn't reachable.
316

PLUGIN METHODS

318       WRITEME
319

HOOKS

321   add_hook
322       WRITEME
323
324   add_backend_hook
325       WRITEME
326

SEE ALSO

328       <http://www.danga.com/mogilefs/>
329
331       This module is Copyright 2003-2004 Brad Fitzpatrick, and copyright
332       2005-2007 Six Apart, Ltd.
333
334       All rights reserved.
335
336       You may distribute under the terms of either the GNU General Public
337       License or the Artistic License, as specified in the Perl README file.
338

WARRANTY

340       This is free software. IT COMES WITHOUT WARRANTY OF ANY KIND.
341

AUTHORS

343       Brad Fitzpatrick <brad@danga.com>
344
345       Brad Whitaker <whitaker@danga.com>
346
347       Mark Smith <marksmith@danga.com>
348
349
350
351perl v5.36.0                      2023-01-20               MogileFS::Client(3)
Impressum