1GANESHA-RADOS-GRACE(8)            NFS-Ganesha           GANESHA-RADOS-GRACE(8)
2
3
4

NAME

6       ganesha-rados-grace - manipulate the shared grace management database
7

SYNOPSIS

9       ganesha-rados-grace [ --cephconf /path/to/ceph.conf ] [--ns namespace] [ --oid obj_id ] [ --pool pool_id ] [ --userid cephuser ] dump|add|start|join|lift|remove|enforce|noenforce|member [ nodeid ... ]
10
11

DESCRIPTION

13       This  tool allows the administrator to directly manipulate the database
14       used by the rados_cluster recovery  backend.  Cluster  nodes  use  that
15       database to indicate their current state in order to coordinate a clus‐
16       ter-wide grace period.
17
18       The first argument should be a command to execute against the database.
19       Any  remaining  arguments represent the nodeids of nodes in the cluster
20       that should be acted upon.
21
22       Most commands will just fail if the grace database is not present.  The
23       exception  to  this rule is the add command which will create the pool,
24       database and namespace if they do not already exist.
25
26       Note that this program  does  not  consult  ganesha.conf.  If  you  use
27       non-default  values for ceph_conf, userid, grace_oid, namespace or pool
28       in your RADOS_KV config block, then they will need  to  passed  in  via
29       command-line options.
30

OPTIONS

32       --cephconf
33
34       Specify  the ceph.conf configuration that should be used (default is to
35       use the normal search path to find one)
36
37       --ns
38
39       Set the RADOS namespace to use within the pool (default is NULL)
40
41       --oid
42
43       Set the object id of  the  grace  database  RADOS  object  (default  is
44       "grace")
45
46       --pool
47
48       Set  the  RADOS  poolid in which the grace database object resides (de‐
49       fault is "nfs-ganesha")
50
51       --userid
52
53       Set the cephx user ID to use when contacting the  cluster  (default  is
54       NULL)
55

COMMANDS

57       dump
58
59       Dump  the  current  status of the grace period database to stdout. This
60       will show the current and recovery epoch serial numbers, as well  as  a
61       list  of hosts currently in the cluster and what flags they have set in
62       their individual records.
63
64       add
65
66       Add the specified hosts to the cluster. This must be  done  before  the
67       given  hosts can take part in the cluster. Attempts to modify the data‐
68       base by cluster hosts that have not yet been added will generally fail.
69       New  hosts are added with the enforcing flag set, as they are unable to
70       hand out new state until their own grace period has been lifted.
71
72       start
73
74       Start a new grace period. This will begin a new  grace  period  in  the
75       cluster  if one is not already active and set the record for the listed
76       cluster hosts as both needing a grace period and  enforcing  the  grace
77       period. If a grace period is already active, then this is equivalent to
78       join.
79
80       join
81
82       Attempt to join an existing grace period. This works  like  start,  but
83       only if there is already an existing grace period in force.
84
85       lift
86
87       Attempt  to  lift  the  current  grace period. This will clear the need
88       grace flags for the listed hosts. If there are no  more  hosts  in  the
89       cluster  that  require a grace period, then it will be fully lifted and
90       the cluster will transition to normal operations.
91
92       remove
93
94       Remove one or more existing hosts from the cluster.  This  will  remove
95       the  listed hosts from the grace database, possibly lifting the current
96       grace period if there are no more hosts that need one.
97
98       enforce
99
100       Set the flag for the given hosts that indicates that they are currently
101       enforcing  the  grace period; not allowing the acquisition of new state
102       by clients.
103
104       noenforce
105
106       Clear the enforcing flag for the given hosts, meaning that those  hosts
107       are now allowing clients to acquire new state.
108
109       member
110
111       Test whether the given hosts are members of the cluster. Returns an er‐
112       ror if any of the hosts are not present in the grace db omap.
113

FLAGS

115       When the dump command is issued,  ganesha-rados-grace  will  display  a
116       list of all of the nodes in the grace database, and any flags they have
117       set.  The flags are as follows:
118
119       E (Enforcing)
120
121       The node is currently enforcing the grace period by rejecting  requests
122       from clients to acquire new state.
123
124       N (Need Grace)
125
126       The  node currently requires a grace period. Generally, this means that
127       the node has clients that need to perform recovery.
128

NODEID ASSIGNMENT

130       Each running ganesha daemon requires a nodeid  string  that  is  unique
131       within  the  cluster.  This can be any value as ganesha treats it as an
132       opaque string. By default, the ganesha daemon will use the hostname  of
133       the node where it is running.
134
135       This  may  not be suitable when running under certain HA clustering in‐
136       frastructure, so it's generally recommended to manually  assign  nodeid
137       values to the hosts in the RADOS_KV config block of ganesha.conf.
138

GANESHA CONFIGURATION

140       The  ganesha daemon will need to be configured with the RecoveryBackend
141       set to rados_cluster. If you use a non-default pool, namespace or  oid,
142       nodeid  then  those  values  will need to be set accordingly in the RA‐
143       DOS_KV config block as well.
144

STARTING A NEW CLUSTER

146       First, add the given cluster nodes to the grace database. Assuming that
147       the nodes in our cluster will have nodeids ganesha-1 through ganesha-3:
148
149       ganesha-rados-grace add ganesha-1 ganesha-2 ganesha-3
150
151       Once this is done, you can start the daemons on each host and they will
152       coordinate to start and lift the grace periods as-needed.
153

ADDING NODES TO A RUNNING CLUSTER

155       After this point, new nodes can then be added to the cluster as  needed
156       using the add command:
157
158       ganesha-rados-grace add ganesha-4
159
160       After  the  node  has  been added, ganesha.nfsd can then be started. It
161       will then request a new grace period as-needed.
162

REMOVING A NODE FROM THE CLUSTER

164       To remove a node from the cluster, first unmount any clients that  have
165       that node mounted (possibly moving them to other servers). Then execute
166       the remove command with the nodeids to be removed from the cluster. For
167       example:
168
169       ganesha-rados-grace remove ganesha-4
170
171       This will remove the ganesha-4's record from the database, and possibly
172       lift the current grace period if one is active and it was the last  one
173       to need it.
174
175
176
177
178                                 Jan 20, 2023           GANESHA-RADOS-GRACE(8)
Impressum