1GANESHA-RADOS-GRACE(8) NFS-Ganesha GANESHA-RADOS-GRACE(8)
2
6 ganesha-rados-grace - manipulate the shared grace management database
7
9 ganesha-rados-grace [ --cephconf /path/to/ceph.conf ] [--ns namespace] [ --oid obj_id ] [ --pool pool_id ] [ --userid cephuser ] dump|add|start|join|lift|remove|enforce|noenforce|member [ nodeid ... ]
10
13 This tool allows the administrator to directly manipulate the database
14 used by the rados_cluster recovery backend. Cluster nodes use that
15 database to indicate their current state in order to coordinate a clus‐
16 ter-wide grace period.
17 The first argument should be a command to execute against the database.
19 Any remaining arguments represent the nodeids of nodes in the cluster
20 that should be acted upon.
21 Most commands will just fail if the grace database is not present. The
23 exception to this rule is the add command which will create the pool,
24 database and namespace if they do not already exist.
25 Note that this program does not consult ganesha.conf. If you use
27 non-default values for ceph_conf, userid, grace_oid, namespace or pool
28 in your RADOS_KV config block, then they will need to passed in via
29 command-line options.
30
32 --cephconf
33 Specify the ceph.conf configuration that should be used (default is to
35 use the normal search path to find one)
36 --ns
38 Set the RADOS namespace to use within the pool (default is NULL)
40 --oid
42 Set the object id of the grace database RADOS object (default is
44 "grace")
45 --pool
47 Set the RADOS poolid in which the grace database object resides (de‐
49 fault is "nfs-ganesha")
50 --userid
52 Set the cephx user ID to use when contacting the cluster (default is
54 NULL)
55
57 dump
58 Dump the current status of the grace period database to stdout. This
60 will show the current and recovery epoch serial numbers, as well as a
61 list of hosts currently in the cluster and what flags they have set in
62 their individual records.
63 add
65 Add the specified hosts to the cluster. This must be done before the
67 given hosts can take part in the cluster. Attempts to modify the data‐
68 base by cluster hosts that have not yet been added will generally fail.
69 New hosts are added with the enforcing flag set, as they are unable to
70 hand out new state until their own grace period has been lifted.
71 start
73 Start a new grace period. This will begin a new grace period in the
75 cluster if one is not already active and set the record for the listed
76 cluster hosts as both needing a grace period and enforcing the grace
77 period. If a grace period is already active, then this is equivalent to
78 join.
79 join
81 Attempt to join an existing grace period. This works like start, but
83 only if there is already an existing grace period in force.
84 lift
86 Attempt to lift the current grace period. This will clear the need
88 grace flags for the listed hosts. If there are no more hosts in the
89 cluster that require a grace period, then it will be fully lifted and
90 the cluster will transition to normal operations.
91 remove
93 Remove one or more existing hosts from the cluster. This will remove
95 the listed hosts from the grace database, possibly lifting the current
96 grace period if there are no more hosts that need one.
97 enforce
99 Set the flag for the given hosts that indicates that they are currently
101 enforcing the grace period; not allowing the acquisition of new state
102 by clients.
103 noenforce
105 Clear the enforcing flag for the given hosts, meaning that those hosts
107 are now allowing clients to acquire new state.
108 member
110 Test whether the given hosts are members of the cluster. Returns an er‐
112 ror if any of the hosts are not present in the grace db omap.
113
115 When the dump command is issued, ganesha-rados-grace will display a
116 list of all of the nodes in the grace database, and any flags they have
117 set. The flags are as follows:
118 E (Enforcing)
120 The node is currently enforcing the grace period by rejecting requests
122 from clients to acquire new state.
123 N (Need Grace)
125 The node currently requires a grace period. Generally, this means that
127 the node has clients that need to perform recovery.
128
130 Each running ganesha daemon requires a nodeid string that is unique
131 within the cluster. This can be any value as ganesha treats it as an
132 opaque string. By default, the ganesha daemon will use the hostname of
133 the node where it is running.
134 This may not be suitable when running under certain HA clustering in‐
136 frastructure, so it's generally recommended to manually assign nodeid
137 values to the hosts in the RADOS_KV config block of ganesha.conf.
138
140 The ganesha daemon will need to be configured with the RecoveryBackend
141 set to rados_cluster. If you use a non-default pool, namespace or oid,
142 nodeid then those values will need to be set accordingly in the RA‐
143 DOS_KV config block as well.
144
146 First, add the given cluster nodes to the grace database. Assuming that
147 the nodes in our cluster will have nodeids ganesha-1 through ganesha-3:
148 ganesha-rados-grace add ganesha-1 ganesha-2 ganesha-3
150 Once this is done, you can start the daemons on each host and they will
152 coordinate to start and lift the grace periods as-needed.
153
155 After this point, new nodes can then be added to the cluster as needed
156 using the add command:
157 ganesha-rados-grace add ganesha-4
159 After the node has been added, ganesha.nfsd can then be started. It
161 will then request a new grace period as-needed.
162
164 To remove a node from the cluster, first unmount any clients that have
165 that node mounted (possibly moving them to other servers). Then execute
166 the remove command with the nodeids to be removed from the cluster. For
167 example:
168 ganesha-rados-grace remove ganesha-4
170 This will remove the ganesha-4's record from the database, and possibly
172 lift the current grace period if one is active and it was the last one
173 to need it.
174 Jan 20, 2023 GANESHA-RADOS-GRACE(8)