1Lucy::Docs::FileLockingU(s3eprm)Contributed Perl DocumenLtuactyi:o:nDocs::FileLocking(3pm)
2
3
4

NAME

6       Lucy::Docs::FileLocking - Manage indexes on shared volumes.
7

DESCRIPTION

9       Normally, index locking is an invisible process.  Exclusive write
10       access is controlled via lockfiles within the index directory and
11       problems only arise if multiple processes attempt to acquire the write
12       lock simultaneously; search-time processes do not ordinarily require
13       locking at all.
14
15       On shared volumes, however, the default locking mechanism fails, and
16       manual intervention becomes necessary.
17
18       Both read and write applications accessing an index on a shared volume
19       need to identify themselves with a unique "host" id, e.g. hostname or
20       ip address.  Knowing the host id makes it possible to tell which
21       lockfiles belong to other machines and therefore must not be removed
22       when the lockfile’s pid number appears not to correspond to an active
23       process.
24
25       At index-time, the danger is that multiple indexing processes from
26       different machines which fail to specify a unique "host" id can delete
27       each others’ lockfiles and then attempt to modify the index at the same
28       time, causing index corruption.  The search-time problem is more
29       complex.
30
31       Once an index file is no longer listed in the most recent snapshot,
32       Indexer attempts to delete it as part of a post-<lucy:Indexer.Commit>
33       cleanup routine.  It is possible that at the moment an Indexer is
34       deleting files which it believes no longer needed, a Searcher
35       referencing an earlier snapshot is in fact using them.  The more often
36       that an index is either updated or searched, the more likely it is that
37       this conflict will arise from time to time.
38
39       Ordinarily, the deletion attempts are not a problem.   On a typical
40       unix volume, the files will be deleted in name only: any process which
41       holds an open filehandle against a given file will continue to have
42       access, and the file won’t actually get vaporized until the last
43       filehandle is cleared.  Thanks to “delete on last close semantics”, an
44       Indexer can’t truly delete the file out from underneath an active
45       Searcher.   On Windows, where file deletion fails whenever any process
46       holds an open handle, the situation is different but still workable:
47       Indexer just keeps retrying after each commit until deletion finally
48       succeeds.
49
50       On NFS, however, the system breaks, because NFS allows files to be
51       deleted out from underneath active processes.  Should this happen, the
52       unlucky read process will crash with a “Stale NFS filehandle”
53       exception.
54
55       Under normal circumstances, it is neither necessary nor desirable for
56       IndexReaders to secure read locks against an index, but for NFS we have
57       to make an exception.  LockFactory’s
58       <lucy:LockFactory.Make_Shared_Lock> method exists for this reason;
59       supplying an IndexManager instance to IndexReader’s constructor
60       activates an internal locking mechanism using
61       <lucy:LockFactory.Make_Shared_Lock> which prevents concurrent indexing
62       processes from deleting files that are needed by active readers.
63
64           use Sys::Hostname qw( hostname );
65           my $hostname = hostname() or die "Can't get unique hostname";
66           my $manager = Lucy::Index::IndexManager->new( host => $hostname );
67
68           # Index time:
69           my $indexer = Lucy::Index::Indexer->new(
70               index   => '/path/to/index',
71               manager => $manager,
72           );
73
74           # Search time:
75           my $reader = Lucy::Index::IndexReader->open(
76               index   => '/path/to/index',
77               manager => $manager,
78           );
79           my $searcher = Lucy::Search::IndexSearcher->new( index => $reader );
80
81       Since shared locks are implemented using lockfiles located in the index
82       directory (as are exclusive locks), reader applications must have write
83       access for read locking to work.  Stale lock files from crashed
84       processes are ordinarily cleared away the next time the same machine –
85       as identified by the "host" parameter – opens another IndexReader. (The
86       classic technique of timing out lock files is not feasible because
87       search processes may lie dormant indefinitely.) However, please be
88       aware that if the last thing a given machine does is crash, lock files
89       belonging to it may persist, preventing deletion of obsolete index
90       data.
91
92
93
94perl v5.38.0                      2023-07-20      Lucy::Docs::FileLocking(3pm)
Impressum