1CEPH-DEPLOY(8) Ceph CEPH-DEPLOY(8)
2
6 ceph-deploy - Ceph deployment tool
7
9 ceph-deploy new [initial-monitor-node(s)]
10 ceph-deploy install [ceph-node] [ceph-node...]
12 ceph-deploy mon create-initial
14 ceph-deploy osd create --data device ceph-node
16 ceph-deploy admin [admin-node][ceph-node...]
18 ceph-deploy purgedata [ceph-node][ceph-node...]
20 ceph-deploy forgetkeys
22
25 ceph-deploy is a tool which allows easy and quick deployment of a Ceph
26 cluster without involving complex and detailed manual configuration. It
27 uses ssh to gain access to other Ceph nodes from the admin node, sudo
28 for administrator privileges on them and the underlying Python scripts
29 automates the manual process of Ceph installation on each node from the
30 admin node itself. It can be easily run on an workstation and doesn't
31 require servers, databases or any other automated tools. With ceph-de‐
32 ploy, it is really easy to set up and take down a cluster. However, it
33 is not a generic deployment tool. It is a specific tool which is de‐
34 signed for those who want to get Ceph up and running quickly with only
35 the unavoidable initial configuration settings and without the overhead
36 of installing other tools like Chef, Puppet or Juju. Those who want to
37 customize security settings, partitions or directory locations and want
38 to set up a cluster following detailed manual steps, should use other
39 tools i.e, Chef, Puppet, Juju or Crowbar.
40 With ceph-deploy, you can install Ceph packages on remote nodes, create
42 a cluster, add monitors, gather/forget keys, add OSDs and metadata
43 servers, configure admin hosts or take down the cluster.
44
46 new
47 Start deploying a new cluster and write a configuration file and
48 keyring for it. It tries to copy ssh keys from admin node to gain
49 passwordless ssh to monitor node(s), validates host IP, creates a clus‐
50 ter with a new initial monitor node or nodes for monitor quorum, a ceph
51 configuration file, a monitor secret keyring and a log file for the new
52 cluster. It populates the newly created Ceph configuration file with
53 fsid of cluster, hostnames and IP addresses of initial monitor members
54 under [global] section.
55 Usage:
57 ceph-deploy new [MON][MON...]
59 Here, [MON] is the initial monitor hostname (short hostname i.e, host‐
61 name -s).
62 Other options like --no-ssh-copykey, --fsid, --cluster-network and
64 --public-network can also be used with this command.
65 If more than one network interface is used, public network setting has
67 to be added under [global] section of Ceph configuration file. If the
68 public subnet is given, new command will choose the one IP from the re‐
69 mote host that exists within the subnet range. Public network can also
70 be added at runtime using --public-network option with the command as
71 mentioned above.
72 install
74 Install Ceph packages on remote hosts. As a first step it installs
75 yum-plugin-priorities in admin and other nodes using passwordless ssh
76 and sudo so that Ceph packages from upstream repository get more prior‐
77 ity. It then detects the platform and distribution for the hosts and
78 installs Ceph normally by downloading distro compatible packages if ad‐
79 equate repo for Ceph is already added. --release flag is used to get
80 the latest release for installation. During detection of platform and
81 distribution before installation, if it finds the distro.init to be
82 sysvinit (Fedora, CentOS/RHEL etc), it doesn't allow installation with
83 custom cluster name and uses the default name ceph for the cluster.
84 If the user explicitly specifies a custom repo url with --repo-url for
86 installation, anything detected from the configuration will be overrid‐
87 den and the custom repository location will be used for installation of
88 Ceph packages. If required, valid custom repositories are also de‐
89 tected and installed. In case of installation from a custom repo a
90 boolean is used to determine the logic needed to proceed with a custom
91 repo installation. A custom repo install helper is used that goes
92 through config checks to retrieve repos (and any extra repos defined)
93 and installs them. cd_conf is the object built from argparse that holds
94 the flags and information needed to determine what metadata from the
95 configuration is to be used.
96 A user can also opt to install only the repository without installing
98 Ceph and its dependencies by using --repo option.
99 Usage:
101 ceph-deploy install [HOST][HOST...]
103 Here, [HOST] is/are the host node(s) where Ceph is to be installed.
105 An option --release is used to install a release known as CODENAME (de‐
107 fault: firefly).
108 Other options like --testing, --dev, --adjust-repos, --no-adjust-repos,
110 --repo, --local-mirror, --repo-url and --gpg-url can also be used with
111 this command.
112 mds
114 Deploy Ceph mds on remote hosts. A metadata server is needed to use
115 CephFS and the mds command is used to create one on the desired host
116 node. It uses the subcommand create to do so. create first gets the
117 hostname and distro information of the desired mds host. It then tries
118 to read the bootstrap-mds key for the cluster and deploy it in the de‐
119 sired host. The key generally has a format of {cluster}.boot‐
120 strap-mds.keyring. If it doesn't finds a keyring, it runs gatherkeys to
121 get the keyring. It then creates a mds on the desired host under the
122 path /var/lib/ceph/mds/ in /var/lib/ceph/mds/{cluster}-{name} format
123 and a bootstrap keyring under /var/lib/ceph/bootstrap-mds/ in
124 /var/lib/ceph/bootstrap-mds/{cluster}.keyring format. It then runs ap‐
125 propriate commands based on distro.init to start the mds.
126 Usage:
128 ceph-deploy mds create [HOST[:DAEMON-NAME]] [HOST[:DAEMON-NAME]...]
130 The [DAEMON-NAME] is optional.
132 mon
134 Deploy Ceph monitor on remote hosts. mon makes use of certain subcom‐
135 mands to deploy Ceph monitors on other nodes.
136 Subcommand create-initial deploys for monitors defined in mon initial
138 members under [global] section in Ceph configuration file, wait until
139 they form quorum and then gatherkeys, reporting the monitor status
140 along the process. If monitors don't form quorum the command will even‐
141 tually time out.
142 Usage:
144 ceph-deploy mon create-initial
146 Subcommand create is used to deploy Ceph monitors by explicitly speci‐
148 fying the hosts which are desired to be made monitors. If no hosts are
149 specified it will default to use the mon initial members defined under
150 [global] section of Ceph configuration file. create first detects plat‐
151 form and distro for desired hosts and checks if hostname is compatible
152 for deployment. It then uses the monitor keyring initially created us‐
153 ing new command and deploys the monitor in desired host. If multiple
154 hosts were specified during new command i.e, if there are multiple
155 hosts in mon initial members and multiple keyrings were created then a
156 concatenated keyring is used for deployment of monitors. In this
157 process a keyring parser is used which looks for [entity] sections in
158 monitor keyrings and returns a list of those sections. A helper is then
159 used to collect all keyrings into a single blob that will be used to
160 inject it to monitors with --mkfs on remote nodes. All keyring files
161 are concatenated to be in a directory ending with .keyring. During this
162 process the helper uses list of sections returned by keyring parser to
163 check if an entity is already present in a keyring and if not, adds it.
164 The concatenated keyring is used for deployment of monitors to desired
165 multiple hosts.
166 Usage:
168 ceph-deploy mon create [HOST] [HOST...]
170 Here, [HOST] is hostname of desired monitor host(s).
172 Subcommand add is used to add a monitor to an existing cluster. It
174 first detects platform and distro for desired host and checks if host‐
175 name is compatible for deployment. It then uses the monitor keyring,
176 ensures configuration for new monitor host and adds the monitor to the
177 cluster. If the section for the monitor exists and defines a monitor
178 address that will be used, otherwise it will fallback by resolving the
179 hostname to an IP. If --address is used it will override all other op‐
180 tions. After adding the monitor to the cluster, it gives it some time
181 to start. It then looks for any monitor errors and checks monitor sta‐
182 tus. Monitor errors arise if the monitor is not added in mon initial
183 members, if it doesn't exist in monmap and if neither public_addr nor
184 public_network keys were defined for monitors. Under such conditions,
185 monitors may not be able to form quorum. Monitor status tells if the
186 monitor is up and running normally. The status is checked by running
187 ceph daemon mon.hostname mon_status on remote end which provides the
188 output and returns a boolean status of what is going on. False means a
189 monitor that is not fine even if it is up and running, while True means
190 the monitor is up and running correctly.
191 Usage:
193 ceph-deploy mon add [HOST]
195 ceph-deploy mon add [HOST] --address [IP]
197 Here, [HOST] is the hostname and [IP] is the IP address of the desired
199 monitor node. Please note, unlike other mon subcommands, only one node
200 can be specified at a time.
201 Subcommand destroy is used to completely remove monitors on remote
203 hosts. It takes hostnames as arguments. It stops the monitor, verifies
204 if ceph-mon daemon really stopped, creates an archive directory mon-re‐
205 move under /var/lib/ceph/, archives old monitor directory in {clus‐
206 ter}-{hostname}-{stamp} format in it and removes the monitor from clus‐
207 ter by running ceph remove... command.
208 Usage:
210 ceph-deploy mon destroy [HOST] [HOST...]
212 Here, [HOST] is hostname of monitor that is to be removed.
214 gatherkeys
216 Gather authentication keys for provisioning new nodes. It takes host‐
217 names as arguments. It checks for and fetches client.admin keyring,
218 monitor keyring and bootstrap-mds/bootstrap-osd keyring from monitor
219 host. These authentication keys are used when new monitors/OSDs/MDS are
220 added to the cluster.
221 Usage:
223 ceph-deploy gatherkeys [HOST] [HOST...]
225 Here, [HOST] is hostname of the monitor from where keys are to be
227 pulled.
228 disk
230 Manage disks on a remote host. It actually triggers the ceph-volume
231 utility and its subcommands to manage disks.
232 Subcommand list lists disk partitions and Ceph OSDs.
234 Usage:
236 ceph-deploy disk list HOST
238 Subcommand zap zaps/erases/destroys a device's partition table and con‐
240 tents. It actually uses ceph-volume lvm zap remotely, alternatively
241 allowing someone to remove the Ceph metadata from the logical volume.
242 osd
244 Manage OSDs by preparing data disk on remote host. osd makes use of
245 certain subcommands for managing OSDs.
246 Subcommand create prepares a device for Ceph OSD. It first checks
248 against multiple OSDs getting created and warns about the possibility
249 of more than the recommended which would cause issues with max allowed
250 PIDs in a system. It then reads the bootstrap-osd key for the cluster
251 or writes the bootstrap key if not found. It then uses ceph-volume
252 utility's lvm create subcommand to prepare the disk, (and journal if
253 using filestore) and deploy the OSD on the desired host. Once pre‐
254 pared, it gives some time to the OSD to start and checks for any possi‐
255 ble errors and if found, reports to the user.
256 Bluestore Usage:
258 ceph-deploy osd create --data DISK HOST
260 Filestore Usage:
262 ceph-deploy osd create --data DISK --journal JOURNAL HOST
264 NOTE:
266 For other flags available, please see the man page or the --help
267 menu on ceph-deploy osd create
268 Subcommand list lists devices associated to Ceph as part of an OSD. It
270 uses the ceph-volume lvm list output that has a rich output, mapping
271 OSDs to devices and other interesting information about the OSD setup.
272 Usage:
274 ceph-deploy osd list HOST
276 admin
278 Push configuration and client.admin key to a remote host. It takes the
279 {cluster}.client.admin.keyring from admin node and writes it under
280 /etc/ceph directory of desired node.
281 Usage:
283 ceph-deploy admin [HOST] [HOST...]
285 Here, [HOST] is desired host to be configured for Ceph administration.
287 config
289 Push/pull configuration file to/from a remote host. It uses push sub‐
290 command to takes the configuration file from admin host and write it to
291 remote host under /etc/ceph directory. It uses pull subcommand to do
292 the opposite i.e, pull the configuration file under /etc/ceph directory
293 of remote host to admin node.
294 Usage:
296 ceph-deploy config push [HOST] [HOST...]
298 ceph-deploy config pull [HOST] [HOST...]
300 Here, [HOST] is the hostname of the node where config file will be
302 pushed to or pulled from.
303 uninstall
305 Remove Ceph packages from remote hosts. It detects the platform and
306 distro of selected host and uninstalls Ceph packages from it. However,
307 some dependencies like librbd1 and librados2 will not be removed be‐
308 cause they can cause issues with qemu-kvm.
309 Usage:
311 ceph-deploy uninstall [HOST] [HOST...]
313 Here, [HOST] is hostname of the node from where Ceph will be unin‐
315 stalled.
316 purge
318 Remove Ceph packages from remote hosts and purge all data. It detects
319 the platform and distro of selected host, uninstalls Ceph packages and
320 purges all data. However, some dependencies like librbd1 and librados2
321 will not be removed because they can cause issues with qemu-kvm.
322 Usage:
324 ceph-deploy purge [HOST] [HOST...]
326 Here, [HOST] is hostname of the node from where Ceph will be purged.
328 purgedata
330 Purge (delete, destroy, discard, shred) any Ceph data from
331 /var/lib/ceph. Once it detects the platform and distro of desired
332 host, it first checks if Ceph is still installed on the selected host
333 and if installed, it won't purge data from it. If Ceph is already unin‐
334 stalled from the host, it tries to remove the contents of
335 /var/lib/ceph. If it fails then probably OSDs are still mounted and
336 needs to be unmounted to continue. It unmount the OSDs and tries to re‐
337 move the contents of /var/lib/ceph again and checks for errors. It also
338 removes contents of /etc/ceph. Once all steps are successfully com‐
339 pleted, all the Ceph data from the selected host are removed.
340 Usage:
342 ceph-deploy purgedata [HOST] [HOST...]
344 Here, [HOST] is hostname of the node from where Ceph data will be
346 purged.
347 forgetkeys
349 Remove authentication keys from the local directory. It removes all the
350 authentication keys i.e, monitor keyring, client.admin keyring, boot‐
351 strap-osd and bootstrap-mds keyring from the node.
352 Usage:
354 ceph-deploy forgetkeys
356 pkg
358 Manage packages on remote hosts. It is used for installing or removing
359 packages from remote hosts. The package names for installation or re‐
360 moval are to be specified after the command. Two options --install and
361 --remove are used for this purpose.
362 Usage:
364 ceph-deploy pkg --install [PKGs] [HOST] [HOST...]
366 ceph-deploy pkg --remove [PKGs] [HOST] [HOST...]
368 Here, [PKGs] is comma-separated package names and [HOST] is hostname of
370 the remote node where packages are to be installed or removed from.
371
373 --address
374 IP address of the host node to be added to the cluster.
375 --adjust-repos
377 Install packages modifying source repos.
378 --ceph-conf
380 Use (or reuse) a given ceph.conf file.
381 --cluster
383 Name of the cluster.
384 --dev Install a bleeding edge built from Git branch or tag (default:
386 master).
387 --cluster-network
389 Specify the (internal) cluster network.
390 --dmcrypt
392 Encrypt [data-path] and/or journal devices with dm-crypt.
393 --dmcrypt-key-dir
395 Directory where dm-crypt keys are stored.
396 --install
398 Comma-separated package(s) to install on remote hosts.
399 --fs-type
401 Filesystem to use to format disk (xfs, btrfs or ext4). Note
402 that support for btrfs and ext4 is no longer tested or recom‐
403 mended; please use xfs.
404 --fsid Provide an alternate FSID for ceph.conf generation.
406 --gpg-url
408 Specify a GPG key url to be used with custom repos (defaults to
409 ceph.com).
410 --keyrings
412 Concatenate multiple keyrings to be seeded on new monitors.
413 --local-mirror
415 Fetch packages and push them to hosts for a local repo mirror.
416 --mkfs Inject keys to MONs on remote nodes.
418 --no-adjust-repos
420 Install packages without modifying source repos.
421 --no-ssh-copykey
423 Do not attempt to copy ssh keys.
424 --overwrite-conf
426 Overwrite an existing conf file on remote host (if present).
427 --public-network
429 Specify the public network for a cluster.
430 --remove
432 Comma-separated package(s) to remove from remote hosts.
433 --repo Install repo files only (skips package installation).
435 --repo-url
437 Specify a repo url that mirrors/contains Ceph packages.
438 --testing
440 Install the latest development release.
441 --username
443 The username to connect to the remote host.
444 --version
446 The current installed version of ceph-deploy.
447 --zap-disk
449 Destroy the partition table and content of a disk.
450
452 ceph-deploy is part of Ceph, a massively scalable, open-source, dis‐
453 tributed storage system. Please refer to the documentation at
454 https://ceph.com/ceph-deploy/docs for more information.
455
457 ceph-mon(8), ceph-osd(8), ceph-volume(8), ceph-mds(8)
458
460 2010-2022, Inktank Storage, Inc. and contributors. Licensed under Cre‐
461 ative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)
462dev Jun 22, 2022 CEPH-DEPLOY(8)