File::RsyncP(3pm)

1File::RsyncP(3)       User Contributed Perl Documentation      File::RsyncP(3)
2
3
4

NAME

6       File::RsyncP - Perl Rsync client
7

SYNOPSIS

9           use File::RsyncP;
10
11           my $rs = File::RsyncP->new({
12                       logLevel   => 1,
13                       rsyncCmd   => "/bin/rsync",
14                       rsyncArgs  => [
15                               "--numeric-ids",
16                               "--perms",
17                               "--owner",
18                               "--group",
19                               "--devices",
20                               "--links",
21                               "--ignore-times",
22                               "--block-size=700",
23                               "--relative",
24                               "--recursive",
25                               "-v",
26                           ],
27                   });
28
29           #
30           # Receive files from remote srcDirectory to local destDirectory
31           # by running rsyncCmd with rsyncArgs.
32           #
33           $rs->remoteStart(1, srcDirectory);
34           $rs->go(destDirectory);
35           $rs->serverClose;
36
37           #
38           # Send files to remote destDirectory from local srcDirectory
39           # by running rsyncCmd with rsyncArgs.
40           #
41           $rs->remoteStart(0, destDirectory);
42           $rs->go(srcDirectory);
43           $rs->serverClose;
44
45           #
46           # Receive files from a remote module to local destDirectory by
47           # connecting to an rsyncd server.  ($module is the name from
48           # /etc/rsyncd.conf.)
49           #
50           my $port = 873;
51           $rs->serverConnect($host, $port);
52           $rs->serverService($module, $authUser, $authPasswd, 0);
53           $rs->serverStart(1, ".");
54           $rs->go(destDirectory);
55           $rs->serverClose;
56
57           #
58           # Get finals stats.  This is a hashref containing elements
59           # totalRead, totalWritten, totalSize, plus whatever the FileIO
60           # module might add.
61           #
62           my $stats = $rs->statsFinal;
63

DESCRIPTION

65       File::RsyncP is a perl implementation of an Rsync client.  It is com‐
66       patible with Rsync 2.5.5 - 2.6.3 (protocol versions 26-28).  It can
67       send or receive files, either by running rsync on the remote machine,
68       or connecting to an rsyncd deamon on the remote machine.
69
70       What use is File::RsyncP?  The main purpose is that File::RsyncP sepa‐
71       rates all file system I/O into a separate module, which can be replaced
72       by any module of your own design.  This allows rsync interfaces to non-
73       filesystem data types (eg: databases) to be developed with relative
74       ease.
75
76       File::RsyncP was initially written to provide an Rsync interface for
77       BackupPC, <http://backuppc.sourceforge.net>.  See BackupPC for program‐
78       ming examples.
79
80       File::RsyncP does not yet provide a command-line interface that mimics
81       native Rsync.  Instead it provides an API that makes it possible to
82       write simple scripts that talk to rsync or rsyncd.
83
84       The File::RsyncP::FileIO module contains the default file system access
85       functions.  File::RsyncP::FileIO may be subclassed or replaced by a
86       custom module to provide access to non-filesystem data types.
87
88       Getting Started
89
90       First some background.  When you run rsync is parses its command-line
91       arguments, then it either connects to a remote rsyncd daemon, or runs
92       an rsync on the remote machine via ssh or rsh.  At this point there are
93       two rsync processes: the one you invoked and the one on the remote
94       machine.  The one on the local machine is called the client, and the
95       one on the remote machine is the server.  One side (either the client
96       or server) will send files and the other will receive files.  The send‐
97       ing rsync generates a file list and sends it to the receiving side.
98       The receiving rsync will fork a child process.
99
100       File::RsyncP does not (yet) have a command-line script that mimics
101       rsync's startup processing.  Think of File::RsyncP as one level below
102       the command-line rsync.  File::RsyncP implements the client side of the
103       connection, and File::RsyncP knows how to run the remote side (eg, via
104       rsh or ssh) or to connect to a remote rsyncd daemon.  File::RsyncP
105       automatically adds the internal --server and --sender options (if nec‐
106       essary) to the options passed to the remote rsync.
107
108       To initiate any rsync session the File::RsyncP->new function should be
109       called.  It takes a hashref of parameters:
110
111       logLevel
112           An integer level of verbosity.  Zero means be quiet, 1 will give
113           some general information, 2 will some output per file, higher val‐
114           ues give more output.  10 will include byte dumps of all data
115           read/written, which will make the log output huge.
116
117       rsyncCmd
118           The command to run the remote peer of rsync.  By default the rsyn‐
119           cArgs are appended to the rsyncCmd to create the complete command
120           before it is run.  This behavior is affected by rsyncCmdType.
121
122           rsyncCmd can either be a single string giving the path of the rsync
123           command to run (eg: /bin/rsync) or a list containing the command
124           and arguments, eg:
125
126               rsyncCmd => [qw(
127                   /bin/ssh -l user host /bin/rsync
128               )],
129
130           or:
131
132               rsyncCmd => ["/bin/ssh", "-l", $user, $host, "/bin/rsync"],
133
134           Also, rsyncCmd can also be set to a code reference (ie: a perl
135           sub).  In this case the code is called without arguments or other
136           processing.  It is up to the perl code you supply to exec() the
137           remote rsync.
138
139           This option is ignored if you are connecting to an rsyncd daemon.
140
141       rsyncCmdType
142           By default the complete remote rsync command is created by taking
143           rsyncCmd and appending rsyncArgs.  This beavhior can be modified by
144           specifying certain values for rsyncCmdType:
145
146           'full'
147               rsyncCmd is taken to be the complete command, including all
148               rsync arguments.  It is the caller's responsibility to build
149               the correct remote rsync command, togheter will all the rsync
150               arguments.  You still need to specify rsyncArgs, so the local
151               File::RsyncP knows how to behave.
152
153           'shell'
154               rsyncArgs are shell escaped before appending to rsyncCmd.
155
156           This option is ignored if you are connecting to an rsyncd daemon.
157
158       rsyncArgs
159           A list of rsync arguments.  The full remote rsync command that is
160           run will be rsyncCmd appended with --server (and optionally
161           --sender if the remote is a sender) and finally all of rsyncArgs.
162
163       protocol_version
164           What we advertize our protocol version to be.  Default is 28.
165
166       logHandler
167           A subroutine reference to a function that handles all the log mes‐
168           sages.  The default is a subroutine that prints the messages to
169           STDERR.
170
171       pidHandler
172           An optional subroutine reference to a function that expects two
173           integers: the pid of the rsync process (ie: the pid on the local
174           machine that is likely ssh) and the child pid when we are receiving
175           files.  If defined, this function is called once when the rsync
176           process is forked, and again when the child is forked during
177           receive.
178
179       fio The file IO object that will handle all the file system IO.  The
180           default is File::RsyncP::FileIO->new.
181
182           This can be replaced with a new module of your choice, or you can
183           subclass File::RsyncP::FileIO.
184
185       timeout
186           Timeout in seconds for IO.  Default is 0, meaning no timeout.  Uses
187           alarm() and it is the caller's responsbility to catch the alarm
188           signal.
189
190       doPartial
191           If set, a partial rsync is done.  This is to support resuming full
192           backups in BackupPC.  When doPartial is set, the --ignore-times
193           option can be set on a per-file basis.  On each file in the file
194           list, File::RsyncP::FileIO->ignoreAttrOnFile() is called on each
195           file, and this returns whether or not attributes should be ignored
196           on that file.  If ignoreAttrOnFile() returns 1 then it's as though
197           --ignore-times was set for that file.
198
199       An example of calling File::RsyncP->new is:
200
201           my $rs = File::RsyncP->new({
202                       logLevel   => 1,
203                       rsyncCmd => ["/bin/rsh", $host,  "-l", $user, "/bin/rsync"],
204                       rsyncArgs  => [
205                               "--numeric-ids",
206                               "--perms",
207                               "--owner",
208                               "--group",
209                               "--devices",
210                               "--links",
211                               "--ignore-times",
212                               "--block-size=700",
213                               "--relative",
214                               "--recursive",
215                               "-v",
216                           ],
217                   });
218
219       A fuller example showing most of the parameters and qw() for the rsyn‐
220       cArgs is:
221
222           my $rs = File::RsyncP->new({
223                       logLevel   => 1,
224                       rsyncCmd => ["/bin/rsh", $host,  "-l", $user, "/bin/rsync"],
225                       rsyncArgs  => [qw(
226                               --numeric-ids
227                               --perms
228                               --owner
229                               --group
230                               --devices
231                               --links
232                               --ignore-times
233                               --block-size=700
234                               --relative
235                               --recursive
236                               -v
237                           )],
238                       logHandler => sub {
239                               my($str) = @_;
240                               print MyHandler "log: $str\n";
241                           };
242                       fio        => File::RsyncP::FileIO->new({
243                                       logLevel   => 1,
244                                   });
245
246                   });
247
248       Talking to a remote Rsync
249
250       File::RsyncP can talk to a remote rsync using this sequence of func‐
251       tions:
252
253       remoteStart(remoteSend, remoteDir)
254           Starts the remote server by executing the command specified in the
255           rsyncCmd parameter to File::RsyncP->new, together with the rsyn‐
256           cArgs.
257
258           If the client is receiving files from the server then remoteSend
259           should be non-zero and remoteDir is the source directory on the
260           remote machine.  If the client is sending files to the remote
261           server then remoteSend should be zero and remoteDir is the destina‐
262           tion directory on the remote machine.  Returns undef on success and
263           non-zero on error.
264
265       go(localDir)
266           Run the client rsync.  localDir is the source directory on the
267           local machine if the client is sending files, or it is the destina‐
268           tion directory on the local machine if the client is receiving
269           files.  Returns undef on success.
270
271       serverClose()
272           Call this after go() to finish up.  Returns undef on success.
273
274       statsFinal()
275           This can be optionally called to pickup the transfer stats.  It
276           returns a hashref containing elements totalRead, totalWritten,
277           totalSize, plus whatever the FileIO module might add.
278
279       abort()
280           Call this function to abort the transfer.
281
282       An example of sending files to a remote rsync is:
283
284           #
285           # Send files to remote destDirectory from local srcDirectory
286           # by running rsyncCmd with rsyncArgs.
287           #
288           $rs->remoteStart(0, destDirectory);
289           $rs->go(srcDirectory);
290           $rs->serverClose;
291
292       An example of receiving files from a remote rsync is:
293
294           #
295           # Receive files from remote srcDirectory to local destDirectory
296           # by running rsyncCmd with rsyncArgs.
297           #
298           $rs->remoteStart(1, srcDirectory);
299           $rs->go(destDirectory);
300           $rs->serverClose;
301
302       Talking to a remote Rsync daemon
303
304       File::RsyncP can connect to a remote Rsync daemon using this sequence
305       of functions:
306
307       serverConnect(host, port)
308           Connect to the Rsync daemon on the given string host and integer
309           port. The port argument is optional and it defaults to 873. On
310           error serverConnect returns a string error message. On success it
311           returns undef.
312
313       serverService(module, authUser, authPasswd, authRequired)
314           Specify which module to use (a "module" is the symbolic name that
315           appears inside "[...]" /etc/rsyncd.conf), the user's credentials
316           (authUser and authPasswd) and whether authorization is mandatory
317           (authRequired). If set to a non-zero value, authRequired ensures
318           that the remote Rsync daemon requires authentication.  If neces‐
319           sary, this is to ensure that you don't connect to an insecure Rsync
320           daemon.  The auth arguments are optional if the selected rsyncd
321           module doesn't require authentication.
322
323           See the rsyncd.conf manual page for more information.  For example,
324           if a host called navajo had a /etc/rsyncd.conf contains these
325           lines:
326
327              [test]
328                      path = /data/test
329                      comment = test module
330                      auth users = craig, celia
331                      secrets file = /etc/rsyncd.secrets
332
333           and /etc/rsyncd.secrets contained:
334
335               craig:xxx
336
337           then you could connect to this rsyncd using:
338
339               $rs->serverConnect("navajo", 873);
340               $rs->serverService("test", "craig", "xxx", 0);
341
342           The value of the authRequired argument doesn't matter in this case.
343
344           On error serverService returns a string error message.  On success
345           it returns undef.
346
347       serverStart(remoteSend, remoteDir)
348           Starts the remote server.  If the client is receiving files from
349           the server then remoteSend should be non-zero.  If the client is
350           sending files to the remote server then remoteSend should be zero.
351           The remoteDir typically starts with the module name, followed by
352           any directory below the module.  Or remoteDir can be just "."  to
353           refer to the top-level module directory.  Returns undef on success.
354
355       go(localDir)
356           Run the client rsync.  localDir is the source directory on the
357           local machine if the client is sending files, or it is the destina‐
358           tion directory on the local machine if the client is receiving
359           files.  Returns undef on success.
360
361       serverClose()
362           Call this after go() to finish up.  Returns undef on success.
363
364       abort()
365           Call this function to abort the transfer.
366
367       An example of sending files to a remote rsyncd daemon is:
368
369           #
370           # Send files to a remote module from a local srcDirectory by
371           # connecting to an rsyncd server.  ($module is the name from
372           # /etc/rsyncd.conf.)
373           #
374           my $port = 873;
375           $rs->serverConnect($host, $port);
376           $rs->serverService($module, $authUser, $authPasswd);
377           $rs->serverStart(0, ".");
378           $rs->go(srcDirectory);
379           $rs->serverClose;
380
381       An example of receiving files from a remote rsyncd daemon is:
382
383           #
384           # Receive files from a remote module to local destDirectory by
385           # connecting to an rsyncd server.  ($module is the name from
386           # /etc/rsyncd.conf.)
387           #
388           my $port = 873;
389           $rs->serverConnect($host, $port);
390           $rs->serverService($module, $authUser, $authPasswd);
391           $rs->serverStart(1, ".");
392           $rs->go(destDirectory);
393           $rs->serverClose;
394

LIMITATIONS

396       The initial version of File::RsyncP (0.10) has a number of limitations:
397
398       ·   File::RsyncP only implements a modest subset of Rsync options and
399           features.  In particular, as of 0.10 only these options are sup‐
400           ported:
401
402                   --numeric-ids
403                   --perms⎪-p
404                   --owner⎪-o
405                   --group⎪-g
406                   --devices⎪D
407                   --links⎪-l
408                   --ignore-times⎪I
409                   --block-size=i
410                   --verbose⎪-v
411                   --recursive⎪-r
412                   --relative⎪-R
413
414           Hardlinks are currently not supported.  Other options that only
415           affect the remote side will work correctly since they are passed to
416           the remote Rsync unchanged.
417
418       ·   Also, --relative semantics are not implemented to match rsync, and
419           the trailing "/" behavior of rsync (meaning directory contents, not
420           the directory itself) are not implemented in File::RsyncP.
421
422       ·   File::RsyncP does not yet provide a command-line interface that
423           mimics native Rsync.
424
425       ·   File::RsyncP might work with slightly earlier versions of Rsync but
426           has not been tested.  It certainly will not work with antique ver‐
427           sions of Rsync.
428
429       ·   File::RsyncP does not compute file deltas (ie: it behaves as though
430           --whole-file is specified) or implement exclude or include options
431           when sending files.  File::RsyncP does handle file deltas and
432           exclude and include options when receiving files.
433
434       ·   File::RsyncP does not yet implement server functionality (acting
435           like the remote end of a connection or a daemon).  Since the proto‐
436           col is relatively symmetric this is not difficult to add, so it
437           should appear in a future version.
438

AUTHOR

440       File::RsyncP::FileList was written by Craig Barratt <cbar‐
441       ratt@users.sourceforge.net> based on rsync 2.5.5.
442
443       Rsync was written by Andrew Tridgell <tridge@samba.org> and Paul Mack‐
444       erras.  It is available under a GPL license.  See
445       http://rsync.samba.org.
446

LICENSE

448       This program is free software; you can redistribute it and/or modify it
449       under the terms of the GNU General Public License as published by the
450       Free Software Foundation; either version 2 of the License, or (at your
451       option) any later version.
452
453       This program is distributed in the hope that it will be useful, but
454       WITHOUT ANY WARRANTY; without even the implied warranty of MER‐
455       CHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
456       Public License for more details.
457
458       You should have received a copy of the GNU General Public License in
459       the LICENSE file along with this program; if not, write to the Free
460       Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
461       02111-1307 USA.
462

NAME

SYNOPSIS

DESCRIPTION

LIMITATIONS

AUTHOR

LICENSE

SEE ALSO