1NVMETCLI(8)                                                        NVMETCLI(8)
2
3
4

NAME

6       nvmetcli - Configure NVMe-over-Fabrics Target.
7

USAGE

9           nvmetcli
10           nvmetcli clear
11           nvmetcli restore [filename.json]
12

DESCRIPTION

14       nvmetcli is a program used for viewing, editing, saving, and starting a
15       Linux kernel NVMe Target, used for an NVMe-over-Fabrics network
16       configuration. It allows an administrator to export a storage resource
17       (such as NVMe devices, files, and volumes) to a local block device and
18       expose them to remote systems based on the NVMe-over-Fabrics
19       specification from http://www.nvmexpress.org.
20
21       nvmetcli is run as root and has two modes:
22
23        1. An interactive configuration shell
24
25        2. Command-line mode which uses an argument
26

BACKGROUND

28       The term NQN used throughout this man page is the NVMe Qualified Name
29       format which an NVMe endpoint (device, subsystem, etc) must follow to
30       guarantee a unique name under the NVMe standard. Any name in a network
31       system setup can be used, but if it does not follow the NQN format, it
32       may not be unique on an NVMe-over-Fabrics network.
33
34       Note that some of the fields set for an NVMe Target port under
35       interactive mode are defined in the "Discovery Log Page" section of
36       NVMe-over-Fabrics specification. Each NVMe Target has a discovery
37       controller mechanism that an NVMe Host can use to determine the NVM
38       subsystems it can access. nvmetcli can be used to add a new record to
39       the discovery controller upon each new subsystem entry and port entry
40       that the newly created subsystem entry binds to (see OPTIONS and
41       EXAMPLES sections). Each NVMe Host only gets to see the discovery
42       entries defined in /subsystems/[NQN NAME]/allowed_hosts and the IP port
43       it is connected to the NVMe Target. An NVMe Host can retrieve these
44       discovery logs via the nvme-cli tool
45       (https://github.com/linux-nvme/nvme-cli).
46

OPTIONS

48       Interactive Configuration Shell
49
50       To start the interactive configuration shell, type nvmetcli on the
51       command-line. nvmetcli interacts with the Linux kernel NVMe Target
52       configfs subsystem starting at base nvmetcli directories /port,
53       /subsystem, and /host. Configuration changes entered by the
54       administrator are made immediately to the kernel target configuration.
55       The following commands can be used while in the interactive
56       configuration shell mode:
57
58       ┌───────────────────────────┬────────────────────────────┐
59       │                           │                            │
60       │cd                         │ Allows to move around the  │
61       │                           │ tree.                      │
62       ├───────────────────────────┼────────────────────────────┤
63       │                           │                            │
64       │ls                         │ Lists contents of current  │
65       │                           │ tree node.                 │
66       ├───────────────────────────┼────────────────────────────┤
67       │                           │                            │
68       │create [NQN name]/[#]      │ Create a new object using  │
69       │                           │ the specified name or      │
70       │                           │ number. If a [NQN          │
71       │                           │ name]/[#] is not           │
72       │                           │ specified, a random entry  │
73       │                           │ will be used.              │
74       ├───────────────────────────┼────────────────────────────┤
75       │                           │                            │
76       │delete [NQN name]/[#]      │ Delete an object with the  │
77       │                           │ specified name or number.  │
78       ├───────────────────────────┼────────────────────────────┤
79       │                           │                            │
80       │set attr                   │ Used under                 │
81       │allow_any_host=[0/1]       │ /subsystems/[NQN name] to  │
82       │                           │ specify if any NVMe Host   │
83       │                           │ can connect to the         │
84       │                           │ subsystem.                 │
85       ├───────────────────────────┼────────────────────────────┤
86       │                           │                            │
87       │set device path=[device    │ Used under                 │
88       │path]                      │ /subsystems/[NQN           
89       │                           │ name]/namespaces to set    │
90       │                           │ the (storage) device to be │
91       │                           │ used.                      │
92       ├───────────────────────────┼────────────────────────────┤
93       │                           │                            │
94       │set device nguid=[string]  │ Used under                 │
95       │                           │ /subsystems/[NQN           
96       │                           │ name]/namespaces to set    │
97       │                           │ the unique id of the       │
98       │                           │ device to the defined      │
99       │                           │ namespace.                 │
100       ├───────────────────────────┼────────────────────────────┤
101       │                           │                            │
102       │enable/disable             │ Used under                 │
103       │                           │ /subsystems/[NQN           
104       │                           │ name]/namespaces to enable │
105       │                           │ and disable the namespace. │
106       ├───────────────────────────┼────────────────────────────┤
107       │                           │                            │
108       │set addr [discovery log    │ Used under /ports/[#] to   │
109       │page field]=[string]       │ create a port which access │
110       │                           │ is allowed. See EXAMPLES   
111       │                           │ for more information.      │
112       ├───────────────────────────┼────────────────────────────┤
113       │                           │                            │
114       │saveconfig [filename.json] │ Save the NVMe Target       │
115       │                           │ configuration in .json     │
116       │                           │ format. Without specifying │
117       │                           │ the filename this will     │
118       │                           │ save as                    │
119       │                           │ /etc/nvmet/config.json.    │
120       │                           │ This file is in JSON       │
121       │                           │ format and can be edited   │
122       │                           │ directly using a prefered  │
123       │                           │ file editor.               │
124       ├───────────────────────────┼────────────────────────────┤
125       │                           │                            │
126       │exit                       │ Quits interactive          │
127       │                           │ configuration shell mode.  │
128       └───────────────────────────┴────────────────────────────┘
129
130       Command Line Mode
131
132       Typing nvmetcli [cmd] on the command-line will execute a command and
133       not enter the interactive configuration shell.
134
135       ┌────────────────────────┬───────────────────────────┐
136       │                        │                           │
137       │restore [filename.json] │ Loads a saved NVMe Target │
138       │                        │ configuration. Without    │
139       │                        │ specifying the filename   │
140       │                        │ this will use             │
141       │                        │ /etc/nvmet/config.json.   │
142       ├────────────────────────┼───────────────────────────┤
143       │                        │                           │
144       │clear                   │ Clears a current NVMe     │
145       │                        │ Target configuration.     │
146       ├────────────────────────┼───────────────────────────┤
147       │                        │                           │
148       │ls                      │ Dumps the current NVMe    │
149       │                        │ Target configuration.     │
150       └────────────────────────┴───────────────────────────┘
151

EXAMPLES

153       Make sure to run nvmetcli as root, the nvmet module is loaded, your
154       devices and all dependent modules are loaded, and configfs is mounted
155       on /sys/kernel/config using:
156
157           mount -t configfs none /sys/kernel/config
158
159       The following section walks through a configuration example.
160
161       •   To get started with the interactive mode and the nvmetcli command
162           prompt, type (in root):
163
164           # ./nvmetcli
165           ...>
166
167       •   Create a subsystem. If you do not specify a name a NQN will be
168           generated, which is probably the best choice. We don’t do it here
169           as the name would be random:
170
171           > cd /subsystems
172           ...> create testnqn
173
174       •   Add access for a specific NVMe Host by it’s NQN:
175
176           ...> cd /hosts
177           ...> create hostnqn
178           ...> cd /subsystems/testnqn
179           ...> set attr allow_any_host=0
180           ...> cd /subsystems/testnqn/allowed_hosts/
181           ...> create hostnqn
182
183       •   Remove access of a subsystem by deleting the Host NQN:
184
185           ...> cd /subsystems/testnqn/allowed_hosts/
186           ...> delete hostnqn
187
188       •   Alternatively this allows any Host to connect to the subsystsem.
189           Only use this in tightly controlled environments:
190
191           ...> cd /subsystems/testnqn/
192           ...> set attr allow_any_host=1
193
194       •   Create a new namespace. If you do not specify a namespace ID the
195           fist unused one will be used:
196
197           ...> cd /subsystems/testnqn/namespaces
198           ...> create 1
199           ...> cd 1
200           ...> set device path=/dev/nvme0n1
201           ...> enable
202
203       Note that in the above setup the device_nguid attribute does not have
204       to be set for correct NVMe Target functionality (but to correctly match
205       a namespace to the exact device upon clear and restore operations, it
206       is advised to set the device_nguid parameter).
207
208       •   Create a loopback port that can be used with nvme-loop module on
209           the same physical machine...
210
211           ...> cd /ports/
212           ...> create 1
213           ...> cd 1/
214           ...> set addr trtype=loop
215           ...> cd subsystems/
216           ...> create testnqn
217
218       •   or create an RDMA (IB, RoCE, iWarp) port using IPv4 addressing.
219           4420 is the IANA assigned default port for NVMe over Fabrics using
220           RDMA:
221
222           ...> cd /ports/
223           ...> create 2
224           ...> cd 2/
225           ...> set addr trtype=rdma
226           ...> set addr adrfam=ipv4
227           ...> set addr traddr=192.168.6.68
228           ...> set addr trsvcid=4420
229           ...> cd subsystems/
230           ...> create testnqn
231
232       •   or create an FC port. traddr is the WWNN/WWPN of the FC port.
233
234           ...> cd /ports/
235           ...> create 3
236           ...> cd 3/
237           ...> set addr trtype=fc
238           ...> set addr adrfam=fc
239           ...> set addr traddr=nn-0x1000000044001123:pn-0x2000000055001123
240           ...> set addr trsvcid=none
241           ...> cd subsystems/
242           ...> create testnqn
243
244       •   Saving the NVMe Target configuration:
245
246           ./nvmetcli
247           ...> saveconfig test.json
248
249       •   Loading an NVMe Target configuration:
250
251             ./nvmetcli restore test.json
252
253       •   Clearing a current NVMe Target configuration:
254
255             ./nvmetcli clear
256

ADDITIONAL INFORMATION

258       nvmetcli has the ability to start and stop the NVMe Target
259       configuration on boot and shutdown through the systemctl Linux utility
260       via a .service file. nvmetcli package comes with nvmet.service which
261       when installed, it can automatically restore the default, saved NVMe
262       Target configuration from /etc/nvmet/config.json. nvmet.service can be
263       installed in directories such as /lib/systemd/system.
264
265       To explicitly enable the service, type:
266
267             systemctl enable nvmet
268
269       To explicitly disable the service, type:
270
271             systemctl disable nvmet
272
273       See also systemctl(1).
274

AUTHORS

276       This man page was written by Jay Freyensee[1]. nvmetcli was originally
277       written by Christoph Hellwig[2].
278

REPORTING BUGS & DEVELOPMENT

280       Please send patches and bug reports to
281       linux-nvme@lists.infradead.org[3] for review and acceptance.
282

LICENSE

284       nvmetcli is licensed under the Apache License, Version 2.0. Software
285       distributed under this license is distributed on an "AS IS" BASIS,
286       WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either expressed or
287       implied.
288

NOTES

290        1. Jay Freyensee
291           mailto:james.p.freyensee@intel.com
292
293        2. Christoph Hellwig
294           mailto:hch@infradead.org
295
296        3. linux-nvme@lists.infradead.org
297           mailto:linux-nvme@lists.infradead.org
298
299
300
301                                  01/19/2023                       NVMETCLI(8)
Impressum