1GANESHA-RADOS-GRACE(8) NFS-Ganesha GANESHA-RADOS-GRACE(8)
2
6 ganesha-rados-grace - manipulate the shared grace management database
7
9 ganesha-rados-grace [--ns namespace] [ --oid obj_id ] [ --pool pool_id ] dump|add|start|join|lift|remove|enforce|noenforce|member [ nodeid ... ]
10
13 This tool allows the administrator to directly manipulate the database
14 used by the rados_cluster recovery backend. Cluster nodes use that
15 database to indicate their current state in order to coordinate a clus‐
16 ter-wide grace period.
17 The first argument should be a command to execute against the database.
19 Any remaining arguments represent the nodeids of nodes in the cluster
20 that should be acted upon.
21 Most commands will just fail if the grace database is not present. The
23 exception to this rule is the add command which will create the pool,
24 database and namespace if they do not already exist.
25
27 --ns
28 Set the RADOS namespace to use within the pool (default is NULL)
30 --oid
32 Set the object id of the grace database RADOS object (default is
34 "grace")
35 --pool
37 Set the RADOS poolid in which the grace database object resides
39 (default is "nfs-ganesha")
40
42 dump
43 Dump the current status of the grace period database to stdout. This
45 will show the current and recovery epoch serial numbers, as well as a
46 list of hosts currently in the cluster and what flags they have set in
47 their individual records.
48 add
50 Add the specified hosts to the cluster. This must be done before the
52 given hosts can take part in the cluster. Attempts to modify the data‐
53 base by cluster hosts that have not yet been added will generally fail.
54 New hosts are added with the enforcing flag set, as they are unable to
55 hand out new state until their own grace period has been lifted.
56 start
58 Start a new grace period. This will begin a new grace period in the
60 cluster if one is not already active and set the record for the listed
61 cluster hosts as both needing a grace period and enforcing the grace
62 period. If a grace period is already active, then this is equivalent to
63 join.
64 join
66 Attempt to join an existing grace period. This works like start, but
68 only if there is already an existing grace period in force.
69 lift
71 Attempt to lift the current grace period. This will clear the need
73 grace flags for the listed hosts. If there are no more hosts in the
74 cluster that require a grace period, then it will be fully lifted and
75 the cluster will transition to normal operations.
76 remove
78 Remove one or more existing hosts from the cluster. This will remove
80 the listed hosts from the grace database, possibly lifting the current
81 grace period if there are no more hosts that need one.
82 enforce
84 Set the flag for the given hosts that indicates that they are currently
86 enforcing the grace period; not allowing the acquisition of new state
87 by clients.
88 noenforce
90 Clear the enforcing flag for the given hosts, meaning that those hosts
92 are now allowing clients to acquire new state.
93 member
95 Test whether the given hosts are members of the cluster. Returns an
97 error if any of the hosts are not present in the grace db omap.
98
100 When the dump command is issued, ganesha-rados-grace will display a
101 list of all of the nodes in the grace database, and any flags they have
102 set. The flags are as follows:
103 E (Enforcing)
105 The node is currently enforcing the grace period by rejecting requests
107 from clients to acquire new state.
108 N (Need Grace)
110 The node currently requires a grace period. Generally, this means that
112 the node has clients that need to perform recovery.
113
115 Each running ganesha daemon requires a nodeid string that is unique
116 within the cluster. This can be any value as ganesha treats it as an
117 opaque string. By default, the ganesha daemon will use the hostname of
118 the node where it is running.
119 This may not be suitable when running under certain HA clustering in‐
121 frastructure, so it's generally recommended to manually assign nodeid
122 values to the hosts in the RADOS_KV config block of ganesha.conf.
123
125 The ganesha daemon will need to be configured with the RecoveryBackend
126 set to rados_cluster. If you use a non-default pool, namespace or oid,
127 nodeid then those values will need to be set accordingly in the
128 RADOS_KV config block as well.
129
131 First, add the given cluster nodes to the grace database. Assuming that
132 the nodes in our cluster will have nodeids ganesha-1 through ganesha-3:
133 ganesha-rados-grace add ganesha-1 ganesha-2 ganesha-3
135 Once this is done, you can start the daemons on each host and they will
137 coordinate to start and lift the grace periods as-needed.
138
140 After this point, new nodes can then be added to the cluster as needed
141 using the add command:
142 ganesha-rados-grace add ganesha-4
144 After the node has been added, ganesha.nfsd can then be started. It
146 will then request a new grace period as-needed.
147
149 To remove a node from the cluster, first unmount any clients that have
150 that node mounted (possibly moving them to other servers). Then execute
151 the remove command with the nodeids to be removed from the cluster. For
152 example:
153 ganesha-rados-grace remove ganesha-4
155 This will remove the ganesha-4's record from the database, and possibly
157 lift the current grace period if one is active and it was the last one
158 to need it.
159 Nov 26, 2019 GANESHA-RADOS-GRACE(8)