1TORUS-2QOS.CONF(5)             OpenIB Management            TORUS-2QOS.CONF(5)
2
3
4

NAME

6       torus-2QoS.conf - Torus-2QoS configuration for OpenSM subnet manager
7

DESCRIPTION

9       The  file  torus-2QoS.conf  contains  configuration information that is
10       specific to the OpenSM routing  engine  torus-2QoS.   Blank  lines  and
11       lines  where  the first non-whitespace character is "#" are ignored.  A
12       token is any contiguous group of non-whitespace characters.  Any tokens
13       on a line following the recognized configuration tokens described below
14       are ignored.
15
16       [torus|mesh] x_radix[m|M|t|T] y_radix[m|M|t|T] z_radix[m|M|t|T]
17              Either torus or mesh must be the first keyword in the configura‐
18              tion,  and  sets  the  topology that torus-2QoS will try to con‐
19              struct.  A 2D topology can be configured by  specifying  one  of
20              x_radix,  y_radix, or z_radix as 1.  An individual dimension can
21              be configured as mesh (open) or torus (looped) by suffixing  its
22              radix specification with one of m, M, t, or T.  Thus, "mesh 3T 4
23              5" and "torus 3 4M 5M" both specify the same topology.
24
25              Note that although torus-2QoS can route mesh fabrics, its  abil‐
26              ity to route around failed components is severely compromised on
27              such fabrics.  A failed fabric component is very likely to cause
28              a disjoint ring; see UNICAST ROUTING in torus-2QoS(8).
29
30       xp_link sw0_GUID sw1_GUID
31       yp_link sw0_GUID sw1_GUID
32       zp_link sw0_GUID sw1_GUID
33       xm_link sw0_GUID sw1_GUID
34       ym_link sw0_GUID sw1_GUID
35       zm_link sw0_GUID sw1_GUID
36              These  keywords  are  used to seed the torus/mesh topology.  For
37              example, "xp_link 0x2000 0x2001" specifies that a link from  the
38              switch with node GUID 0x2000 to the switch with node GUID 0x2001
39              would point in the positive x direction, while  "xm_link  0x2000
40              0x2001"  specifies  that  a  link from the switch with node GUID
41              0x2000 to the switch with node GUID 0x2001 would  point  in  the
42              negative  x  direction.   All the link keywords for a given seed
43              must specify the same "from" switch.
44
45              In general, it is not necessary to configure both  the  positive
46              and negative directions for a given coordinate; either is suffi‐
47              cient.  However, the algorithm used for topology discovery needs
48              extra information for torus dimensions of radix four (see TOPOL‐
49              OGY DISCOVERY in torus-2QoS(8)).  For such cases both the  posi‐
50              tive and negative coordinate directions must be specified.
51
52              Based  on  the  topology  specified  via the torus/mesh keyword,
53              torus-2QoS will detect and log when  it  has  insufficient  seed
54              configuration.
55
56       x_dateline position
57       y_dateline position
58       z_dateline position
59              In  order  for  torus-2QoS to provide the guarantee that path SL
60              values do not change under any conditions for which it can still
61              route  the fabric, its idea of dateline position must not change
62              relative to physical switch locations.   The  dateline  keywords
63              provide the means to configure such behavior.
64
65              The  dateline for a torus dimension is always between the switch
66              with coordinate 0 and the switch  with  coordinate  radix-1  for
67              that  dimension.   By default, the common switch in a torus seed
68              is taken as the origin of the coordinate system used to describe
69              switch  location.  The position parameter for a dateline keyword
70              moves the origin (and hence the dateline) the  specified  amount
71              relative to the common switch in a torus seed.
72
73       next_seed
74              If  any  of  the  switches  used  to specify a seed were to fail
75              torus-2QoS would be unable to complete topology  discovery  suc‐
76              cessfully.   The  next_seed keyword specifies that the following
77              link and dateline keywords apply to a new seed specification.
78
79              For maximum resiliency, no seed  specification  should  share  a
80              switch  with any other seed specification.  Multiple seed speci‐
81              fications should  use  dateline  configuration  to  ensure  that
82              torus-2QoS  can  grant path SL values that are constant, regard‐
83              less of which seed was used to initiate topology discovery.
84
85       portgroup_max_ports max_ports
86              This keyword specifies the maximum  number  of  parallel  inter-
87              switch  links,  and  also  the  maximum number of host ports per
88              switch, that torus-2QoS can accommodate.  The default  value  is
89              16.   Torus-2QoS  will log an error message during topology dis‐
90              covery if this parameter needs to be increased.  If this keyword
91              appears multiple times, the last instance prevails.
92
93              Note  that  the  switch management port (switch port 0) gets put
94              into the same port group with the host ports, so if you have  16
95              host  ports  per switch, portgroup_max_ports would need to be at
96              least 17.
97
98       port_order p1 p2 p3 ...
99              This keyword specifies the order in which CA ports on a destina‐
100              tion  switch  are visited when computing routes. When the fabric
101              contains switches connected with multiple parallel links, routes
102              are  distributed in a round-robin fashion across such links, and
103              so changing the order that CA ports are visited changes the dis‐
104              tribution of routes across such links.  This may be advantageous
105              for some specific traffic patterns.
106
107              The default is to visit CA ports in  increasing  port  order  on
108              destination switches.
109
110              Duplicate values in the list will be ignored.
111
112       max_changes max
113              This  keyword  specifies  the  maximum  number  of torus changes
114              reported.  The default value is 32.
115

EXAMPLE

117
118       # Look for a 2D (since x radix is one) 4x5 torus.
119       torus 1 4 5
120
121       # y is radix-4 torus dimension, need both
122       # ym_link and yp_link configuration.
123       yp_link 0x200000 0x200005  # sw @ y=0,z=0 -> sw @ y=1,z=0
124       ym_link 0x200000 0x20000f  # sw @ y=0,z=0 -> sw @ y=3,z=0
125
126       # z is not radix-4 torus dimension, only need one of
127       # zm_link or zp_link configuration.
128       zp_link 0x200000 0x200001  # sw @ y=0,z=0 -> sw @ y=0,z=1
129
130       next_seed
131
132       yp_link 0x20000b 0x200010  # sw @ y=2,z=1 -> sw @ y=3,z=1
133       ym_link 0x20000b 0x200006  # sw @ y=2,z=1 -> sw @ y=1,z=1
134       zp_link 0x20000b 0x20000c  # sw @ y=2,z=1 -> sw @ y=2,z=2
135
136       y_dateline -2  # Move the dateline for this seed
137       z_dateline -1  # back to its original position.
138
139       # If OpenSM failover is configured, for maximum resiliency
140       # one instance should run on a host attached to a switch
141       # from the first seed, and another instance should run
142       # on a host attached to a switch from the second seed.
143       # Both instances should use this torus-2QoS.conf to ensure
144       # path SL values do not change in the event of SM failover.
145
146       # port_order defines the order on which the ports would be
147       # chosen for routing.
148       port_order 7 10 8 11 9 12 25 28 26 29 27 30
149
150

FILES

152       /etc/rdma/torus-2QoS.conf
153              Default torus-2QoS config file.
154

SEE ALSO

156       opensm(8), torus-2QoS(8).
157
158
159
160OpenIB                          January 4, 2013             TORUS-2QOS.CONF(5)
Impressum