1ibd(7D) Devices ibd(7D)
2
6 ibd - Infiniband IPoIB device driver
7
9 /dev/ibd*
10
13 The ibd driver implements the IETF IP over Infiniband protocol and pro‐
14 vides IPoIB service for all IBA ports present in the system.
15 The ibd driver is a multi-threaded, loadable, clonable, STREAMS hard‐
18 ware driver supporting the connectionless Data Link Provider Interface,
19 dlpi(7P)). The ibd driver provides basic support for the IBA Unreliable
20 Datagram Queue Pair hardware. Functions include QP initialization,
21 frame transmit and receive, multicast and promiscuous mode support, and
22 statistics reporting.
23 Use the cloning, character-special device /dev/ibd to access all ibd
26 devices installed within the system.
27 The ibd driver is dependent on GLD, a loadable kernel module that pro‐
30 vides the ibd driver with the DLPI and STREAMS functionality required
31 of a LAN driver. Except as noted in the Application Programming Inter‐
32 face section of this manpage, see gld(7D) for more details on the prim‐
33 itives supported by the driver. The GLD module is located at /ker‐
34 nel/misc/sparcv9/gld on 64 bit systems and at /kernel/misc/gld on 32
35 bit systems.
36 The ibd driver expects certain configuration of the IBA fabric prior to
39 operation (which also implies the SM must be active and managing the
40 fabric). Specifically, the IBA multicast group representing the IPv4
41 limited broadcast address 255.255.255.255 (also defined as broadcast-
42 GID in IETF documents) must be created prior to initializing the
43 device. IBA properties (including mtu, qkey and sl) of this group is
44 used by the driver to create any other IBA multicast group as
45 instructed by higher level (IP) software. The driver probes for the
46 existance of this broadcast-GID during attach(9E).
47
49 The values returned by the driver in the DL_INFO_ACK primitive in
50 response to your DL_INFO_REQ are:
51 o Maximum SDU is the MTU associated with the broadcast-GID
53 group, less the 4 byte IPoIB header.
54 o Minimum SDU is 0.
56 o dlsap address length is 22.
58 o MAC type is DL_IB.
60 o The sap length value is -2, meaning the physical address
62 component is followed immediately by a 2-byte sap component
63 within the DLSAP address.
64 o Broadcast address value is the MAC address consisting of the
66 4 bytes of QPN 00:FF:FF:FF prepended to the IBA multicast
67 address of the broadcast-GID.
68 Due to the nature of link address definition for IPoIB, the
70 DL_SET_PHYS_ADDR_REQ DLPI primitive is not supported.
71 In the transmit case for streams that have been put in raw
73 mode via the DLIOCRAW ioctl, the DLPI application must
74 prepend the 20 byte IPoIB destination address to the data it
75 wants to transmit over-the-wire. In the receive case, appli‐
76 cations receive the IP/ARP datagram along with the IETF
77 defined 4 byte header.
78
80 This section describes warning messages that might be generated by the
81 driver. Please note that while the format of these messages may be mod‐
82 ified in future versions, the same general information will be pro‐
83 vided.
84 While joining IBA multicast groups corresponding to IP multicast groups
87 as part of multicast promiscuous operations as required by IP multicast
88 routers, or as part of running snoop(1M), it is possible that joins to
89 some multicast groups can fail due to inherent resource constraints in
90 the IBA components. In such cases, warning message similar to the fol‐
91 lowing appear in the system log, indicating the interface on which the
92 failure occurred:
93 NOTICE: ibd0: Could not get list of IBA multicast groups
95 NOTICE: ibd0: IBA promiscuous mode missed multicast group
96 NOTICE: ibd0: IBA promiscuous mode missed new multicast gid
97 Also, if the IBA SM indicates that multicast trap support is suspended
101 or unavailable, the system log contains a message similar to:
102 NOTICE: ibd0: IBA multicast support degraded due to
104 unavailability of multicast traps
105 And when the SM indicates trap support is restored:
109 NOTICE: ibd0: IBA multicast support restored due to
111 availability of multicast traps
112 Additionally, if the IBA link transitions to an unavailable state (that
116 is, the IBA link state becomes "Down," "Initialize" or "Armed") and
117 then becomes active again, the driver tries to rejoin previously joined
118 groups if required. Failure to rejoin multicast groups will trigger
119 messages like:
120 NOTICE: ibd0: Failure on port up to rejoin multicast gid
122 If the corresponding HCA port is in the unavailable state defined above
126 when initializing an ibd interface using ifconfig(1M), a message is
127 emitted by the driver:
128 NOTICE: ibd0: Port is not active
130 Further, as described above, if the broadcast-GID is not found, or the
134 associated MTU is higher than what the HCA port can support, the fol‐
135 lowing messages are printed to the system log:
136 NOTICE: ibd0: IPoIB broadcast group absent
138 NOTICE: ibd0: IPoIB broadcast group MTU 4096 greater than port's
139 maximum MTU 2048
140 In all cases of these reported problems when running ifconfig(1M), it
144 should be checked that IBA cabling is intact, an SM is running on the
145 fabric, and the broadcast-GID with appropriate properties has been cre‐
146 ated in the IBA partition.
147
149 The IPoIB service comes preconfigured on all HCA ports in the system.
150 To turn the service off, or back on after turning it off, refer to doc‐
151 umentation in cfgadm_ib(1M).
152
154 /dev/ibd* special character device
155 /kernel/drv/ib.conf configuration file to start IPoIB service
158 /kernel/drv/sparcv9/ibd 64-bit SPARC device driver
161 /kernel/drv/amd64/ibd 64-bit x86 device driver
164 /kernel/drv/ibd 32-bit x86 device driver
167
170 cfgadm(1M), cfgadm_ib(1M), ifconfig(1M), syslogd(1M), gld(7D), ib(7D),
171 kstat(7D), streamio(7I), dlpi(7P), attributes(5), attach(9E)
172
174 IBD is a GLD-based driver and provides the statistics described by
175 gld(7D). Note that valid received packets not accepted by any stream
176 (long) will increase when IBD transmits broadcast IP packets. This hap‐
177 pens because the infiniband hardware copies and loops back the trans‐
178 mitted broadcast packets to the source. These packets are discarded by
179 GLD and are recorded as 'unknowns'.
180SunOS 5.11 20 Jul 2007 ibd(7D)