1Lucy::Docs::FileLockingU(s3eprm)Contributed Perl DocumenLtuactyi:o:nDocs::FileLocking(3pm)
2
6 Lucy::Docs::FileLocking - Manage indexes on shared volumes.
7
9 Normally, index locking is an invisible process. Exclusive write
10 access is controlled via lockfiles within the index directory and
11 problems only arise if multiple processes attempt to acquire the write
12 lock simultaneously; search-time processes do not ordinarily require
13 locking at all.
14 On shared volumes, however, the default locking mechanism fails, and
16 manual intervention becomes necessary.
17 Both read and write applications accessing an index on a shared volume
19 need to identify themselves with a unique "host" id, e.g. hostname or
20 ip address. Knowing the host id makes it possible to tell which
21 lockfiles belong to other machines and therefore must not be removed
22 when the lockfile’s pid number appears not to correspond to an active
23 process.
24 At index-time, the danger is that multiple indexing processes from
26 different machines which fail to specify a unique "host" id can delete
27 each others’ lockfiles and then attempt to modify the index at the same
28 time, causing index corruption. The search-time problem is more
29 complex.
30 Once an index file is no longer listed in the most recent snapshot,
32 Indexer attempts to delete it as part of a post-<lucy:Indexer.Commit>
33 cleanup routine. It is possible that at the moment an Indexer is
34 deleting files which it believes no longer needed, a Searcher
35 referencing an earlier snapshot is in fact using them. The more often
36 that an index is either updated or searched, the more likely it is that
37 this conflict will arise from time to time.
38 Ordinarily, the deletion attempts are not a problem. On a typical
40 unix volume, the files will be deleted in name only: any process which
41 holds an open filehandle against a given file will continue to have
42 access, and the file won’t actually get vaporized until the last
43 filehandle is cleared. Thanks to “delete on last close semantics”, an
44 Indexer can’t truly delete the file out from underneath an active
45 Searcher. On Windows, where file deletion fails whenever any process
46 holds an open handle, the situation is different but still workable:
47 Indexer just keeps retrying after each commit until deletion finally
48 succeeds.
49 On NFS, however, the system breaks, because NFS allows files to be
51 deleted out from underneath active processes. Should this happen, the
52 unlucky read process will crash with a “Stale NFS filehandle”
53 exception.
54 Under normal circumstances, it is neither necessary nor desirable for
56 IndexReaders to secure read locks against an index, but for NFS we have
57 to make an exception. LockFactory’s
58 <lucy:LockFactory.Make_Shared_Lock> method exists for this reason;
59 supplying an IndexManager instance to IndexReader’s constructor
60 activates an internal locking mechanism using
61 <lucy:LockFactory.Make_Shared_Lock> which prevents concurrent indexing
62 processes from deleting files that are needed by active readers.
63 use Sys::Hostname qw( hostname );
65 my $hostname = hostname() or die "Can't get unique hostname";
66 my $manager = Lucy::Index::IndexManager->new( host => $hostname );
67 # Index time:
69 my $indexer = Lucy::Index::Indexer->new(
70 index => '/path/to/index',
71 manager => $manager,
72 );
73 # Search time:
75 my $reader = Lucy::Index::IndexReader->open(
76 index => '/path/to/index',
77 manager => $manager,
78 );
79 my $searcher = Lucy::Search::IndexSearcher->new( index => $reader );
80 Since shared locks are implemented using lockfiles located in the index
82 directory (as are exclusive locks), reader applications must have write
83 access for read locking to work. Stale lock files from crashed
84 processes are ordinarily cleared away the next time the same machine –
85 as identified by the "host" parameter – opens another IndexReader. (The
86 classic technique of timing out lock files is not feasible because
87 search processes may lie dormant indefinitely.) However, please be
88 aware that if the last thing a given machine does is crash, lock files
89 belonging to it may persist, preventing deletion of obsolete index
90 data.
91perl v5.38.0 2023-07-20 Lucy::Docs::FileLocking(3pm)