1FileList(3)           User Contributed Perl Documentation          FileList(3)
2
3
4

NAME

6       File::RsyncP::FileList - Perl interface to rsync file list encoding and
7       decoding.
8

SYNOPSIS

10           use File::RsyncP::FileList;
11
12           $fileList = File::RsyncP::FileList->new({
13               preserve_uid        => 1,
14               preserve_gid        => 1,
15               preserve_links      => 1,
16               preserve_devices    => 1,
17               preserve_hard_links => 0,
18               always_checksum     => 0,
19               remote_version      => 26,
20           });
21
22           # decoding an incoming file list
23           while ( !$fileList->decodeDone && !$fileList->fatalError ) {
24               $data .= readMoreDataFromRemoteRsync();
25               $bytesDone = $fileList->decode($data);
26               $data = substr($data, $bytesDone) if ( $bytesDone > 0 );
27           }
28           $fileList->clean;
29
30           # create (encode) a file list
31           $fileList->encode({
32                   name  => $filePath,
33                   dev   => $stat[0],
34                   inode => $stat[1],
35                   mode  => $stat[2],
36                   uid   => $stat[4],
37                   gid   => $stat[5],
38                   rdev  => $stat[6],
39                   mtime => $stat[9],
40               });
41           $data = $fileList->encodeData;
42
43           # get file information, for file number 5:
44           $fileInfo = $fileList->get(5);
45
46           # utility functions
47           $numberOfFiles = $fileList->count;
48           $gotFatalError = $fileList->fatalError;
49

DESCRIPTION

51       The File::RsyncP::FileList module is used to encode and decode file
52       lists in using the same format at Rsync.
53
54       The sender side of Rsync creates a list of all the files the are going
55       to be sent.  This list is sent in a compact format to the receiver
56       side.  Each side then sorts the list and removes duplicate entries.
57       From this point on, all files are referred to by their integer index
58       into the sorted file list.
59
60       A new file list object is created by calling
61       File::RsyncP::FileList->new.  An object can be used to decode or encode
62       a file list.  There is no mechanism to reset the state of a file list:
63       you should create a new object each time you need to do a new decode or
64       encode.
65
66       The new() function takes a hashref of options, which correspond to
67       various rsync command-line switches.  These must exactly match the
68       arguments to the remote rsync, otherwise the file list format will not
69       be compatible and decoding will fail.
70
71           $fileList = File::RsyncP::FileList->new({
72               preserve_uid        => 1,       # --owner
73               preserve_gid        => 1,       # --group
74               preserve_links      => 1,       # --links
75               preserve_devices    => 1,       # --devices
76               preserve_hard_links => 0,       # --hard-links
77               always_checksum     => 0,       # --checksum
78               remote_version      => 26,      # remote protocol version
79           });
80
81   Decoding
82       The decoding functions take a stream of bytes from the remote rsync and
83       convert them into an internal data structure.  Rather than store the
84       file list as a native perl list of hashes (which occupies too much
85       memory for large file lists), the same internal data structure as rsync
86       is used.  Individual file list entries can be returned with the get()
87       function.
88
89       File list data read from the remote rsync should be passed to the
90       decode() function.  The data may be read and processed in arbitrary
91       sized chunks.  The decode() function returns how many bytes were
92       actually processed.  It is the caller's responsbility to remove that
93       number of bytes from the input argument, preserving the remaining bytes
94       for the next call to decode().  The decodeDone() function returns true
95       when the file list is complete.  The fatalError() function returns true
96       if there was a non-recoverable error while decoding.
97
98       The clean() function needs to be called after the file list decode is
99       complete.  The clean() function sorts the file list and removes
100       repeated entries.  Skipping this step will produce unexpected results:
101       since files are referred to using integers, each side will refer to
102       different files is the file lists are not sorted and purged in exactly
103       the same manner.
104
105       A typical decode loop looks like:
106
107           while ( !$fileList->decodeDone && !$fileList->fatalError ) {
108               $data .= readMoreDataFromRemoteRsync();
109               $bytesDone = $fileList->decode($data);
110               $data = substr($data, $bytesDone) if ( $bytesDone > 0 );
111           }
112           $fileList->clean;
113
114       After clean() is called, the number of files in the file list can be
115       found by calling count().  Files can be fetched by calling the get()
116       function, with an index from 0 to count()-1:
117
118           $fileInfo = $fileList->get(5);
119
120       The get() function returns a hashref with various entries:
121
122               name      path name of the file (relative to rsync dir):
123                         equal to dirname/basename
124               basename  file name, without directory
125               dirname   directory where file resides
126               sum       file MD4 checksum (only present if --checksum specified)
127               uid       file user id
128               gid       file group id
129               mode      file mode
130               mtime     file modification time
131               size      file length
132               dev       device number on which file resides
133               inode     file inode
134               link      link contents if the file is a sym link
135               rdev      major/minor device number if file is char/block special
136
137       Various fields will only have valid values if the corresponding options
138       are set (eg: uid if preserve_uid is set, dev and inode if
139       preserve_hard_links is set etc).
140
141       For example, to dump out each of hash you could do this:
142
143           use Data::Dumper;
144           my $count = $fileList->count;
145           for ( my $i = 0 ; $i < $count ; $i++ ) {
146               print("File $i is:\n");
147               print Dumper($fileList->get($i));
148           }
149
150   Encoding
151       The encode() function is used to build a file list in preparation for
152       encoding and sending a file list to a remote rsync.  The encode()
153       function takes a hashref argument with the parameters for one file.  It
154       should be called once for each file.  The parameter names are the same
155       as those returned by get().
156
157       In this example the matching stat() values are shown:
158
159           $fileList->encode({
160                   name  => $filePath,
161                   dev   => $stat[0],
162                   inode => $stat[1],
163                   mode  => $stat[2],
164                   uid   => $stat[4],
165                   gid   => $stat[5],
166                   rdev  => $stat[6],
167                   size  => $stat[7],
168                   mtime => $stat[9],
169               });
170
171       It is not necessary to specify basename and dirname; these are
172       extracted from name.  You only need to specify the parameters that
173       match the options given to new().  You can also specify sum and link as
174       necessary.
175
176       To compute the encoded file list data the encodeData() function should
177       be called.  It can be called every time encode() is called, or once at
178       the end of all the encode() calls.  It returns the encoded data that
179       should be sent to the remote rsync:
180
181           $data = $fileList->encodeData;
182
183       It is recommended that encodeData() be called frequently to avoid the
184       need to allocate large internal buffers to hold the entire encoded file
185       list.  Since encodeData() does not know when the last file has been
186       encoded, it is the caller's responsbility to add the final null byte
187       (eg: pack("C", 0)) to the data to indicate the end of the file list
188       data.
189
190       After all the file list entries are processed you should call clean():
191
192           $fileList->clean;
193
194       This ensures that each side (sender/receiver) has identical sorted file
195       lists.
196
197   Utility functions
198       The count() function returns the total number of files in the internal
199       file list (either decoded or encoded).
200
201       The fatalError() function returns true if a fatal error has occured
202       during file decoding.  It should be called in the decode loop to make
203       sure no error has occured.
204

AUTHOR

206       File::RsyncP::FileList was written by Craig Barratt
207       <cbarratt@users.sourceforge.net> based on rsync 2.5.5.
208
209       Rsync was written by Andrew Tridgell <tridge@samba.org> and Paul
210       Mackerras.  It is available under a GPL license.  See
211       http://rsync.samba.org
212

LICENSE

214       This program is free software; you can redistribute it and/or modify it
215       under the terms of the GNU General Public License as published by the
216       Free Software Foundation; either version 2 of the License, or (at your
217       option) any later version.
218
219       This program is distributed in the hope that it will be useful, but
220       WITHOUT ANY WARRANTY; without even the implied warranty of
221       MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
222       General Public License for more details.
223
224       You should have received a copy of the GNU General Public License in
225       the LICENSE file along with this program; if not, write to the Free
226       Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
227       02111-1307 USA.
228

SEE ALSO

230       See <http://perlrsync.sourceforge.net> for File::RsyncP's SourceForge
231       home page.
232
233       See File::RsyncP and File::RsyncP::FileIO for more precise examples of
234       using File::RsyncP::FileList.
235
236       Also see BackupPC's lib/BackupPC/Xfer/RsyncFileIO.pm for other
237       examples.
238
239
240
241perl v5.32.0                      2020-09-29                       FileList(3)
Impressum