1cfgadm_ib(1M) System Administration Commands cfgadm_ib(1M)
2
6 cfgadm_ib - InfiniBand hardware specific commands for cfgadm
7
9 /usr/sbin/cfgadm -f [-y | -n] [-v] -c function ap_id...
10 /usr/sbin/cfgadm [-f] [-y | -n] [-v] [-o hardware_options]
13 -x hardware_function ap_id...
14 /usr/sbin/cfgadm -v [-a] [-s listing_option] [-] [ap_id | ap_type...]
17 /usr/sbin/cfgadm -v -h [ap_id]...
20
23 The InfiniBand hardware specific library /usr/lib/cfgadm/ib.so.1 pro‐
24 vides the functionality for administering its fabric through the
25 cfgadm(1M) utility. cfgadm operates on attachment points. See
26 cfgadm(1M).
27 An InfiniBand (IB) device is enumerated by the IB nexus driver, ib(7D),
30 based on the services from the IB Device Manager (IBDM).
31 The IB nexus driver creates and initializes five types of child device
34 nodes:
35 o IB Port devices
37 o IB HCA service (HCA_SVC) devices
39 o IB Virtual Physical Point of Attachment (VPPA) devices
41 o I/O Controller (IOC)
43 o IB Pseudo devices
45 See ib(7D) for details on enumeration of IB Port, IB VPPA, and IB
48 HCA_SVC devices. For additional information on IBDM, see ibdm(7D). See
49 ib(4) for details on IB Pseudo devices.
50 For IB administration, two types of static attachment point are created
53 for the fabric administration as seen by the given host. There is one
54 static attachment point ib and all IB devices (either an IOC, Port,
55 VPPA, HCA_SVC, or a Pseudo device) in the fabric are represented as
56 dynamic attachment points based off of it. There is another static
57 attachment point for each Host Channel Adapter (HCA) in the host based
58 on its node Globally Unique Identifier (GUID) value.
59 Attachment points are named through ap_ids. There are two types of
62 ap_ids: logical and physical. The physical ap_id is based on the physi‐
63 cal path name. For the IB fabric it is /devices/ib:fabric. The logical
64 ap_id is a shorter, and has a more user friendly name.
65 The static ap_id for the IB fabric is ib. The IB devices are dynamic
68 attachment points and have no physical ap_id. The logical ap_id of an
69 IOC contains its GUID, ib::IOC-GUID. An example of an IOC ap_id is
70 ib::80020123456789a. The logical ap_id of a Pseudo device, see ib(4)
71 for details, is of the format ib::driver_name,unit-address. An example
72 of a pseudo ap_id would be ib::sdp,0 where "sdp" is the driver name and
73 "0" is its unit-address property. The logical ap_id of Port, VPPA and
74 HCA_SVC device contains its Partition Key (P_Key), Port GUID / Node
75 GUID and a communication service-name. The format of ap_id is as below:
76 Port device
78 ib::PORT_GUID,0,service-name
80 VPPA device
83 ib::PORT_GUID,P_Key,service-name
85 HCA_SVC device
88 ib::HCA_GUID,0,servicename
90 The Partition Key (P_Key) is 0 for Port and HCA_SVC devices. The P_Key
94 helps determine the partition to which this port belongs for a VPPA
95 device node. A port might have more than one P_Key. An example of a
96 VPPA device logical ap_id point is ib::80245678,ffff,ipib. The port-
97 GUID is 80245678, the P_Key is 0xffff, and the service name is ipib.
98 The service-name information is obtained from the file /ker‐
99 nel/drv/ib.conf which contains service-name strings. The HCA's logical
100 ap_id contains its node GUID value, hca:HCA-GUID. An example is
101 hca:21346543210a987.
102 A listing of the IB attachment points includes information on all IB
105 devices (IOC, VPPA, HCA_SVC, Pseudo, and Port devices seen by the IBDM
106 and the IB nexus driver) in the fabric even if they are not seen by the
107 host and configured for use.
108 The following shows a listing of five IB devices (two IOC, one VPPA,
111 one Port, one HCA_SVC) and one HCA:
112 example# cfgadm -al
114 Ap_Id Type Receptacle Occupant Condition
115 hca:21346543210a987 IB-HCA connected configured ok
116 ib IB-FABRIC connected configured ok
117 ib::80020123456789a IB-IOC connected configured ok
118 ib::802abc9876543 IB-IOC connected unconfigured unknown
119 ib::80245678,ffff,ipib IB-VPPA connected configured ok
120 ib::12245678,0,nfs IB-PORT connected configured ok
121 ib::21346543,0,hnfs IB-HCA_SVC connected configured ok
122 ib::sdp,0 IB-PSEUDO connected configured ok
123 The ap_id ib::802abc9876543 shows an IOC device that is not yet config‐
128 ured by the host for use or had been previously offlined by an explicit
129 cfgadm -c unconfigure
131 operation. The distinction was made by the information displayed under
136 the Condition column. The IB device with a zero P_Key and HCA GUID is a
137 HCA_SVC device. Refer to cfgadm(1M) for more information regarding
138 listing attachment points.
139 The receptacle state for attachment points have the following meanings:
142 connected
144 For an IOC/VPPA/Port/Pseudo/HCA_SVC device, connected implies that
146 it has been seen by the host. The device might not have been con‐
147 figured for use by Solaris.
148 For a HCA attachment point, connected implies that it has been con‐
150 figured and is in use.
151 All IB ap_ids are always shown as connected.
153 The occupant state for attachment points have the following meanings:
157 configured
159 The IB device, and the HCA ap_id, are configured and usable by
161 Solaris.
162 unconfigured
165 The IB device at the ap_id was explicitly offlined using cfgadm -c
167 unconfigure, was not successfully configured. This could be because
168 it wasn not successfully configuref for use with Solaris (no
169 driver, or a device problem), or because it was never configured
170 for use by the IB nexus driver.
171 The unconfigured operation is not supported for the HCA attachment
173 point. The IB static apid, ib, is shown unconfigured if the system
174 has no IB hardware.
175 The attachment point conditions are:
179 failed
181 Not used.
183 failing
186 Not used.
188 ok
191 Normal state. Ready for use.
193 unknown
196 This state is only valid for IB device that have been probed by
198 IBDM but not yet configured for use by Solaris. It is also shown
199 for devices that have been explicitly offlined by a cfgadm -c
200 unconfigure operation. This condition does not apply to a HCA
201 attachment point.
202 unusable
205 Not used.
207
210 The following options are supported:
211 -c function
213 The IB hardware specific library supports two generic commands
215 (functions). These commands are not supported on the static attach‐
216 ment points (that is, the HCA ap_ids and the IB static ib ap_id).
217 The following generic commands are supported:
219 configure
221 Configure the IB device to be used by Solaris.
223 unconfigure
226 Unconfigure the IB device. If successful, cfgadm reports the
228 condition of this ap_id as unknown.
229 -f
233 Not supported.
235 -h ap_id
238 Obtain IB specific help for an IB attachment point.
240 -l
243 List the state and condition of IB attachment points. The -l option
245 works as described in cfgadm(1M).
246 When paired with the -a option, displays the dynamic attachment
248 points as well (IOC, VPPA, Port, Pseudo, and HCA_SVC devices).
249 When paired with -v option, displays verbose data about the ap_ids.
251 For an IOC, the Info field in the
252 cfgadm -avl
254 output displays the following information: VendorID, IOCDeviceID,
257 DeviceVersion, SubsystemVendorID, SubsystemID, Class, Subclass,
258 Protocol, ProtocolVersion and IDString from the IOCControllerPro‐
259 file. If the ID string isn't provided then nothing is displayed in
260 its place. These fields are defined in the InfiniBand Specification
261 Volume 1 (http://www.infinibandta.org).
262 For a VPPA, Port, or HCA_SVC device the Info field in the cfgadm
264 -lav display shows the service name information to which this
265 device is bound. If no such information exists, nothing is dis‐
266 played.
267 For a Pseudo device cfgadm -alv displays the driver name and its
269 unit-address information. For a HCA the verbose listing displays
270 the VendorID, ProductID of the HCA, number of ports it has, and the
271 PortGUID value of its ports. See EXAMPLES.
272 -o hardware_option
275 This option is not currently defined.
277 -s listing_option
280 Attachment points of class ib can be listed by using the select
282 sub-option. Refer to the cfgadm(1M) man page for more information.
283 -x hardware_function
286 Perform a hardware specific function. Note that the name can not be
288 more than 4 characters long.
289 The following hardware specific functions are supported:
291 add_service -ocomm=[port|vppa|hca_svc],service=name
293 This hardware specific function is supported on the static IB
295 attachment point. It can be used to add a new service to /ker‐
296 nel/drv/ib.conf file and to update the ib(7D) driver.
297 You must use the service=name option to indicate the new ser‐
299 vice to be added. You must use the option
300 comm=[port|vppa|hca_svc] option to add the name service to
301 either port-svc-list or to the hca-svc-list in the /ker‐
302 nel/drv/ib.conf file. See EXAMPLES.
303 delete_service -ocomm=[port|vppa|hca_svc],service=name
306 This hardware specific function is supported on the static IB
308 attachment point only. It can be used to delete an existing
309 service from the /kernel/drv/ib.conf file and also from the
310 ib(7D) driver's data base. You must use the service=name option
311 to indicate which service to delete. You must use the
312 comm=[port|vppa|hca_svc] option to delete this service from the
313 port-svc-list, vppa-svc-list, or vppa-svc-list of the /ker‐
314 nel/drv/ib.conf file. See EXAMPLES.
315 list_clients
318 Supported on HCA attachment points. Displays all the kernel IB
320 clients using this HCA. It also displays the respective ap_ids
321 of these kernel IB clients and if they have opened an alternate
322 HCA device. See EXAMPLES.
323 .
325 If a given kernel IB client does not have a valid ap_id then a
327 - is displayed in that column.
328 list_services
331 This hardware specific function is supported on the static IB
333 attachment point only. It lists all the Port and VPPA services
334 as read from the /kernel/drv/ib.conf file. See EXAMPLES.
335 unconfig_clients
338 This hardware specific function is supported on the static HCA
340 attachment point only. It can be used to unconfigure all IB
341 kernel clients of this given HCA. Only IB kernel clients that
342 do not have an alternate HCA are unconfigured. See EXAMPLES.
343 update_ioc_config
346 This hardware specific function is supported on static ib
348 attachment point and the IOC attachment points. For the ib
349 APID, this function updates properties of all the IOC device
350 nodes. For the IOC APID, this function updates the properties
351 of specified IOC device node. This command updates the port-
352 list, port-entries, service-id, and service-name IOC node prop‐
353 erties .
354 See ib(7D).
356 update_pkey_tbls
359 Supported on the static ib attachment point. Updates the PKEY
361 information inside IBTL. IBTL re-reads the P_Key tables for all
362 the ports on each HCA present on the host.
363 See ibtl(7D).
365
369 Example 1 Listing the State and Condition of IB Devices
370 The following command lists the state and condition of IB devices on
373 the system. It only shows the static attachment points.
374 example# cfgadm
377 hca:21346543210a987 IB-HCA connected configured ok
378 ib IB-FABRIC connected configured ok
379 The -a option lists all attachment points. The following example uses
384 the -a option and lists all attachment points:
385 example# cfgadm -a
388 hca:21346543210a987 IB-HCA connected configured ok
389 ib IB-FABRIC connected configured ok
390 ib::80020123456789a IB-IOC connected unconfigured ok
391 ib::80245678,ffff,ipib IB-VPPA connected configured ok
392 ib::21346543,0,hnfs IB-HCA_SVC connected configured ok
393 ib::12245678,0,nfs IB-PORT connected configured ok
394 ib::sdp,0 IB-PSEUDO connected configured ok
395 Example 2 Listing the Verbose Status of a IB VPPA Device
399 The following command lists the verbose status of a IB VPPA device:
402 example# cfgadm -alv ib::80245678,ffff,ipib
405 Ap_Id Receptacle Occupant Condition Information
406 When Type Busy Phys_Id
407 ib::80245678,ffff,ipib connected configured ok ipib
408 unavailable IB-VPPA n /devices/ib:fabric::80245678,ffff,ipib
409 A verbose listing of an IOC shows additional information. The following
414 command shows a verbose listing:
415 example# cfgadm -alv ib::80020123456789a
418 Ap_Id Receptacle Occupant Condition Information
419 When Type Busy Phys_Id
420 ib::80020123456789a connected configured ok VID: 0xeaea
421 DEVID: 0xeaea VER: 0x5 SUBSYS_VID: 0x0 SUBSYS_ID: 0x0 CLASS: 0xffff
422 SUBCLASS: 0xff PROTO: 0xff PROTOVER: 0x1 ID_STRING: Sample Host Adapter
423 unavailable IB-IOC n /devices/ib:fabric::80020123456789a
424 A verbose listing of a Pseudo device shows:
429 example# cfgadm -alv ib::sdp,0
432 Ap_Id Receptacle Occupant Condition Information
433 When Type Busy Phys_Id
434 ib::sdp,0 connected configured ok Driver = "sd
435 p" Unit-address = "0"
436 unavailable IB-PSEUDO n /devices/ib:fabric::sdp,0
437 A verbose listing of a HCA shows:
442 example# cfgadm -alv hca:21346543210a987
445 Ap_Id Receptacle Occupant Condition Information
446 When Type Busy Phys_Id
447 hca:21346543210a987 connected configured ok VID: 0x15b3,
448 PID: 0x5a44, #ports: 0x2, port1 GUID: 0x80245678, port2 GUID: 0x80245679
449 unavailable IB-HCA n /devices/ib:21346543210a987
450 You can obtain more user-friendly output if you specify these following
455 cfgadm class and field selection options: -s
456 "select=class(ib),cols=ap_id:info"
457 The following command displays only IB ap_ids. The output only includes
461 the ap_id and Information fields.
462 # cfgadm -al -s "cols=ap_id:info" ib::80245678,ffff,ipib
466 Ap_Id Information
467 ib::80245678,ffff,ipib ipib
468 Example 3 Unconfiguring an Existing IB IOC
472 The following command unconfigures the IB IOC attached to
475 ib::80020123456789a, then displays the status of the ap_id:
476 # cfgadm -c unconfigure ib::80020123456789a
479 Unconfigure the device: /devices/ib:fabric::80020123456789a
480 This operation will suspend activity on the IB device
481 Continue (yes/no)?
482 Enter: y
487 IB device unconfigured successfully.
490 # cfgadm -al ib::80020123456789a
491 Ap_Id Type Receptacle Occupant Condition
492 ib::80020123456789 IB-IOC connected unconfigured unknown
493 #
494 The condition unknown implies that the device node doesn't exist any‐
499 more and this IB device's existence is known only to the IB Device Man‐
500 ager.
501 Example 4 Configuring an IB IOC
504 The following series of commands configures an IB device attached to
507 ib::80020123456789a:
508 # cfgadm -yc configure ib::80020123456789a
511 # cfgadm -al ib::80020123456789a
512 Ap_Id Type Receptacle Occupant Condition
513 ib::80020123456789a IB-IOC connected configured ok
514 Example 5 Listing All Kernel IB Clients of a HCA
518 The following command lists all kernel IB clients of an HCA attached to
521 hca:21346543210a987:
522 # cfgadm -x list_clients hca:21346543210a987
525 Attachment Point Clients Alternate HCA
526 ib::80020123456789a ioc1 Yes
527 ib::80245678,ffff,ipib ipib No
528 ib::21346543,0,hnfs hnfs No
529 - ibdm No
530 - ibmf No
531 Example 6 Adding a Port Service
535 The following command adds a new Port service called srp:
538 # cfgadm -o comm=port,service=srp -x add_service ib
541 Example 7 Deleting a VPPA Service
545 The following command deletes the ibd VPPA service ibd:
548 # cfgadm -o comm=vppa,service=ipib -x delete_service ib
551 Example 8 Listing Port, VPPA, HCA_SVC Services
555 The following command lists all Port, VPPA, and HCA_SVC services:
558 # cfgadm -x list_services ib
561 Port communication services:
562 srp
563 VPPA communication services:
565 ipib
566 nfs
567 HCA_SVC communication services:
569 hnfs
570 Example 9 Reprobing IOC Devices
574 The following command reprobes all IOC device nodes.
577 # cfgadm -x update_ioc_config ib
580 This operation can update properties of IOC devices.
581 Continue (yes/no)?
582 Enter: y
584 #
586 Example 10 Unconfiguring All Kernel Clients of a HCA
590 The following command unconfigures all kernel clients of a HCA
593 # cfgadm -x unconfig_clients hca:21346543
596 This operation will unconfigure clients of this HCA.
597 Continue (yes/no)?
598 Enter: y
600
604 /usr/lib/cfgadm/ib.so.1
605 Hardware-specific library for generic InfiniBand device administra‐
607 tion
608
611 See attributes(5) for descriptions of the following attributes:
612 ┌─────────────────────────────┬─────────────────────────────┐
617 │ ATTRIBUTE TYPE │ ATTRIBUTE VALUE │
618 ├─────────────────────────────┼─────────────────────────────┤
619 │Availability │SUNWcsl │
620 └─────────────────────────────┴─────────────────────────────┘
621
623 cfgadm(1M), config_admin(3CFGADM), libcfgadm(3LIB), ib(4),
624 attributes(5), ib(7D), ibdm(7D), ibtl(7D)
625 InfiniBand Specification Volume 1 (http://www.infinibandta.org)
628
630 Apart from the listing (cfgadm -l or cfgadm -x list_clients), only the
631 superuser can execute any functions on an attachment point.
632SunOS 5.11 2 Jul 2008 cfgadm_ib(1M)