1MogileFS::Client(3) User Contributed Perl Documentation MogileFS::Client(3)
2
3
4
6 MogileFS::Client - Client library for the MogileFS distributed file
7 system.
8
10 use MogileFS::Client;
11
12 # create client object w/ server-configured namespace
13 # and IPs of trackers
14 $mogc = MogileFS::Client->new(domain => "foo.com::my_namespace",
15 hosts => ['10.0.0.2:7001', '10.0.0.3:7001']);
16
17 # create a file
18 # mogile is a flat namespace. no paths.
19 $key = "image_of_userid:$userid";
20 # must be configured on server
21 $class = "user_images";
22 $fh = $mogc->new_file($key, $class);
23
24 print $fh $data;
25
26 unless ($fh->close) {
27 die "Error writing file: " . $mogc->errcode . ": " . $mogc->errstr;
28 }
29
30 # Find the URLs that the file was replicated to.
31 # May change over time.
32 @urls = $mogc->get_paths($key);
33
34 # no longer want it?
35 $mogc->delete($key);
36
38 This module is a client library for the MogileFS distributed file
39 system. The class method 'new' creates a client object against a
40 particular mogilefs tracker and domain. This object may then be used to
41 store and retrieve content easily from MogileFS.
42
44 new
45 $client = MogileFS::Client->new( %OPTIONS );
46
47 Creates a new MogileFS::Client object.
48
49 Returns MogileFS::Client object on success, or dies on failure.
50
51 OPTIONS:
52
53 hosts
54 Arrayref of 'host:port' strings to connect to as backend trackers
55 in this client.
56
57 domain
58 String representing the mogile domain which this MogileFS client is
59 associated with. (All create/delete/fetch operations will be
60 performed against this mogile domain). See the mogadm shell command
61 and its 'domain' category of operations for information on
62 manipulating the list of possible domains on a MogileFS system.
63
64 reload
65 $mogc->reload( %OPTIONS )
66
67 Re-init the object, like you'd just reconstructed it with 'new', but
68 change it in-place instead. Useful if you have a system which reloads
69 a config file, and you want to update a singleton $mogc handle's config
70 value.
71
72 last_tracker
73 Returns a scalar of form "ip:port", representing the last mogilefsd
74 'tracker' server which was talked to.
75
76 errstr
77 Returns string representation of the last error that occurred. It
78 includes the error code (same as method 'errcode') and a space before
79 the optional English error message.
80
81 This isn't necessarily guaranteed to reset after a successful
82 operation. Only call it after another operation returns an error.
83
84 errcode
85 Returns an error code. Not a number, but a string identifier (e.g.
86 "no_domain") which is stable for use in error handling logic.
87
88 This isn't necessarily guaranteed to reset after a successful
89 operation. Only call it after another operation returns an error.
90
91 force_disconnect
92 Forces the client to disconnect from the tracker, causing it to
93 reconnect when the next request is made. It will reconnect to a
94 different tracker if possible. A paranoid application may wish to do
95 to this before retrying a failed command, on the off chance that
96 another tracker may be working better.
97
98 readonly
99 $is_readonly = $mogc->readonly
100 $mogc->readonly(1)
101
102 Getter/setter to mark this client object as read-only. Purely a local
103 operation/restriction, doesn't do a network operation to the mogilefsd
104 server.
105
106 new_file
107 $mogc->new_file($key)
108 $mogc->new_file($key, $class)
109 $mogc->new_file($key, $class, $content_length)
110 $mogc->new_file($key, $class, $content_length , $opts_hashref)
111
112 Start creating a new filehandle with the given key, and option given
113 class and options.
114
115 Returns a filehandle you should then print to, and later close to
116 complete the operation. NOTE: check the return value from close! If
117 your close didn't succeed, the file didn't get saved!
118
119 $opts_hashref can contain keys:
120
121 fid Explicitly specify the fid number to use, rather than it being
122 automatically allocated.
123
124 create_open_args
125 Hashref of extra key/value pairs to send to mogilefsd in
126 create_open phase.
127
128 create_close_args
129 Hashref of extra key/value pairs to send to mogilefsd in
130 create_close phase.
131
132 largefile
133 Use MogileFS::ClientHTTPFile which will not load the entire file
134 into memory like the default MogileFS::NewHTTPFile but requires
135 that the storage node HTTP servers support the Content-Range header
136 in PUT requests and is a little slower.
137
138 edit_file
139 $mogc->edit_file($key, $opts_hashref)
140
141 Edit the file with the the given key.
142
143 NOTE: edit_file is currently EXPERIMENTAL and not recommended for
144 production use. MogileFS is primarily designed for storing files for
145 later retrieval, rather than editing. Use of this function may lead to
146 poor performance and, until it has been proven mature, should be
147 considered to also potentially cause data loss.
148
149 NOTE: use of this function requires support for the DAV 'MOVE' verb and
150 partial PUT (i.e. Content-Range in PUT) on the back-end storage servers
151 (e.g. apache with mod_dav).
152
153 Returns a seekable filehandle you can read/write to. Calling this
154 function may invalidate some or all URLs you currently have for this
155 key, so you should call ->get_paths again afterwards if you need them.
156
157 On close of the filehandle, the new file contents will replace the
158 previous contents (and again invalidate any existing URLs).
159
160 By default, the file contents are preserved on open, but you may
161 specify the overwrite option to zero the file first. The seek position
162 is at the beginning of the file, but you may seek to the end to append.
163
164 $opts_hashref can contain keys:
165
166 overwrite
167 The edit will overwrite the file, equivalent to opening with '>'.
168 Default: false.
169
170 read_file
171 $mogc->read_file($key)
172
173 Read the file with the the given key.
174
175 Returns a seekable filehandle you can read() from. Note that you cannot
176 read line by line using <$fh> notation.
177
178 Takes the same options as get_paths (which is called internally to get
179 the URIs to read from).
180
181 store_file
182 $mogc->store_file($key, $class, $fh_or_filename[, $opts_hashref])
183
184 Wrapper around new_file, print, and close.
185
186 Given a key, class, and a filehandle or filename, stores the file
187 contents in MogileFS. Returns the number of bytes stored on success,
188 undef on failure.
189
190 $opts_hashref can contain keys for new_file, and also the following:
191
192 chunk_size
193 Number of bytes to read and write and write at once out of the
194 larger file. Defaults to 8192 bytes. Increasing this can increase
195 performance at the cost of more memory used while uploading the
196 file. Note that this mostly helps when using largefile => 1
197
198 store_content
199 $mogc->store_content($key, $class, $content[, $opts]);
200
201 Wrapper around new_file, print, and close. Given a key, class, and
202 file contents (scalar or scalarref), stores the file contents in
203 MogileFS. Returns the number of bytes stored on success, undef on
204 failure.
205
206 get_paths
207 @paths = $mogc->get_paths($key)
208 @paths = $mogc->get_paths($key, $no_verify_bool); # old way
209 @paths = $mogc->get_paths($key, { noverify => $bool }); # new way
210
211 Given a key, returns an array of all the locations (HTTP URLs) that the
212 file has been replicated to.
213
214 noverify
215 If the "no verify" option is set, the mogilefsd tracker doesn't
216 verify that the first item returned in the list is up/alive.
217 Skipping that check is faster, so use "noverify" if your
218 application can do it faster/smarter. For instance, when giving
219 Perlbal a list of URLs to reproxy to, Perlbal can intelligently
220 find one that's alive, so use noverify and get out of mod_perl or
221 whatever as soon as possible.
222
223 zone
224 If the zone option is set to 'alt', the mogilefsd tracker will use
225 the alternative IP for each host if available, while constructing
226 the paths.
227
228 pathcount
229 If the pathcount option is set to a positive integer greater than
230 2, the mogilefsd tracker will attempt to return that many different
231 paths (if available) to the same file. If not present or out of
232 range, this value defaults to 2.
233
234 get_file_data
235 $dataref = $mogc->get_file_data($key)
236
237 Wrapper around get_paths & LWP, which returns scalarref of file
238 contents in a scalarref.
239
240 Don't use for large data, as it all comes back to you in one string.
241
242 delete
243 $mogc->delete($key);
244
245 Delete a key from MogileFS.
246
247 rename
248 $mogc->rename($oldkey, $newkey);
249
250 Rename file (key) in MogileFS from oldkey to newkey. Returns true on
251 success, failure otherwise.
252
253 file_debug
254 my $info_gob = $mogc->file_debug(fid => $fid);
255 ... or ...
256 my $info_gob = $mogc->file_debug(key => $key);
257
258 Thoroughly search for any database notes about a particular fid.
259 Searchable by raw fidid, or by domain and key. Use sparingly. Command
260 hits the master database numerous times, and if you're using it in
261 production something is likely very wrong.
262
263 To be used with troubleshooting broken/odd files and errors from
264 mogilefsd.
265
266 file_info
267 my $fid = $mogc->file_info($key, { devices => 0 });
268
269 Used to return metadata about a file. Returns the domain, class,
270 expected length, devcount, etc. Optionally device ids (not paths) can
271 be returned as well.
272
273 Should be used for informational purposes, and not usually for
274 dynamically serving files.
275
276 list_keys
277 $keys = $mogc->list_keys($prefix, $after[, $limit]);
278 ($after, $keys) = $mogc->list_keys($prefix, $after[, $limit]);
279
280 Used to get a list of keys matching a certain prefix.
281
282 $prefix specifies what you want to get a list of. $after is the item
283 specified as a return value from this function last time you called it.
284 $limit is optional and defaults to 1000 keys returned.
285
286 In list context, returns ($after, $keys). In scalar context, returns
287 arrayref of keys. The value $after is to be used as $after when you
288 call this function again.
289
290 When there are no more keys in the list, you will get back undef or an
291 empty list.
292
293 foreach_key
294 $mogc->foreach_key( %OPTIONS, sub { my $key = shift; ... } );
295 $mogc->foreach_key( prefix => "foo:", sub { my $key = shift; ... } );
296
297 Functional interface/wrapper around list_keys.
298
299 Given some %OPTIONS (currently only one, "prefix"), calls your callback
300 for each key matching the provided prefix.
301
302 update_class
303 $mogc->update_class($key, $newclass);
304
305 Update the replication class of a pre-existing file, causing the file
306 to become more or less replicated.
307
308 set_pref_ip
309 $mogc->set_pref_ip({ "10.0.0.2" => "10.2.0.2" });
310
311 Weird option for old, weird network architecture. Sets a mapping table
312 of preferred alternate IPs, if reachable. For instance, if trying to
313 connect to 10.0.0.2 in the above example, the module would instead try
314 to connect to 10.2.0.2 quickly first, then then fall back to 10.0.0.2
315 if 10.2.0.2 wasn't reachable.
316
318 WRITEME
319
321 add_hook
322 WRITEME
323
324 add_backend_hook
325 WRITEME
326
328 <http://www.danga.com/mogilefs/>
329
331 This module is Copyright 2003-2004 Brad Fitzpatrick, and copyright
332 2005-2007 Six Apart, Ltd.
333
334 All rights reserved.
335
336 You may distribute under the terms of either the GNU General Public
337 License or the Artistic License, as specified in the Perl README file.
338
340 This is free software. IT COMES WITHOUT WARRANTY OF ANY KIND.
341
343 Brad Fitzpatrick <brad@danga.com>
344
345 Brad Whitaker <whitaker@danga.com>
346
347 Mark Smith <marksmith@danga.com>
348
349
350
351perl v5.32.1 2021-01-27 MogileFS::Client(3)