1NM-CLOUD-SETUP(8) Automatic Network Configuratio NM-CLOUD-SETUP(8)
2
6 nm-cloud-setup - Overview of Automatic Network Configuration in Cloud
7
9 When running a virtual machine in a public cloud environment, it is
10 desirable to automatically configure the network of that VM. In simple
11 setups, the VM only has one network interface and the public cloud
12 supports automatic configuration via DHCP, DHCP6 or IPv6 autoconf.
13 However, the virtual machine might have multiple network interfaces, or
14 multiple IP addresses and IP subnets on one interface which cannot be
15 configured via DHCP. Also, the administrator may reconfigure the
16 network while the machine is running. NetworkManager's nm-cloud-setup
17 is a tool that automatically picks up such configuration in cloud
18 environments and updates the network configuration of the host.
19 Multiple cloud providers are supported. See the section called
21 “SUPPORTED CLOUD PROVIDERS”.
22
24 The goal of nm-cloud-setup is to be configuration-less and work
25 automatically. All you need is to opt-in to the desired cloud providers
26 (see the section called “ENVIRONMENT VARIABLES”) and run
27 /usr/libexec/nm-cloud-setup.
28 Usually this is done by enabling the nm-cloud-setup.service systemd
30 service and let it run periodically. For that there is both a
31 nm-cloud-setup.timer systemd timer and a NetworkManager dispatcher
32 script.
33
35 nm-cloud-setup configures the network by fetching the configuration
36 from the well-known meta data server of the cloud provider. That means,
37 it already needs the network configured to the point where it can reach
38 the meta data server. Commonly that means, that a simple connection
39 profile is activated that possibly uses DHCP to get the primary IP
40 address. NetworkManager will create such a profile for ethernet devices
41 automatically if it is not configured otherwise via "no-auto-default"
42 setting in NetworkManager.conf. One possible alternative may be to
43 create such an initial profile with nmcli device connect "$DEVICE" or
44 nmcli connection add type ethernet ....
45 By setting the user-data org.freedesktop.nm-cloud-setup.skip=yes on the
47 profile, nm-cloud-setup will skip the device.
48 nm-cloud-setup modifies the run time configuration akin to nmcli device
50 modify. With this approach, the configuration is not persisted and only
51 preserved until the device disconnects.
52 /usr/libexec/nm-cloud-setup
54 The binary /usr/libexec/nm-cloud-setup does most of the work. It
55 supports no command line arguments but can be configured via
56 environment variables. See the section called “ENVIRONMENT VARIABLES”
57 for the supported environment variables.
58 By default, all cloud providers are disabled unless you opt-in by
60 enabling one or several providers. If cloud providers are enabled, the
61 program tries to fetch the host's configuration from a meta data server
62 of the cloud via HTTP. If configuration could be not fetched, no cloud
63 provider are detected and the program quits. If host configuration is
64 obtained, the corresponding cloud provider is successfully detected.
65 Then the network of the host will be configured.
66 It is intended to re-run nm-cloud-setup every time when the
68 configuration (maybe) changes. The tool is idempotent, so it should be
69 OK to also run it more often than necessary. You could run
70 /usr/libexec/nm-cloud-setup directly. However it may be preferable to
71 restart the nm-cloud-setup systemd service instead or use the timer or
72 dispatcher script to run it periodically (see below).
73 nm-cloud-setup.service systemd unit
75 Usually /usr/libexec/nm-cloud-setup is not run directly, but only by
76 systemctl restart nm-cloud-setup.service. This ensures that the tool
77 only runs once at any time. It also allows to integrate with the
78 nm-cloud-setup systemd timer, and to enable/disable the service via
79 systemd.
80 As you need to set environment variable to configure nm-cloud-setup
82 binary, you can do so via systemd override files. Try systemctl edit
83 nm-cloud-setup.service.
84 nm-cloud-setup.timer systemd timer
86 /usr/libexec/nm-cloud-setup is intended to run whenever an update is
87 necessary. For example, during boot when when changing the network
88 configuration of the virtual machine via the cloud provider.
89 One way to do this, is by enabling the nm-cloud-setup.timer systemd
91 timer with systemctl enable --now nm-cloud-setup.timer.
92 /usr/lib/NetworkManager/dispatcher.d/90-nm-cloud-setup.sh
94 There is also a NetworkManager dispatcher script that will run for
95 example when an interface is activated by NetworkManager. Together with
96 the nm-cloud-setup.timer systemd timer this script is to automatically
97 pick up changes to the network.
98 The dispatcher script will do nothing, unless the systemd service is
100 enabled. To use the dispatcher script you should therefor run systemctl
101 enable nm-cloud-setup.service once.
102
104 The following environment variables are used to configure
105 /usr/libexec/nm-cloud-setup. You may want to configure them with a
106 drop-in for the systemd service. For example by calling systemctl edit
107 nm-cloud-setup.service and configuring [Service] Environment=, as
108 described in systemd.exec(5) manual.
109 • NM_CLOUD_SETUP_LOG: control the logging verbosity. Set it to one of
111 TRACE, DEBUG, INFO, WARN, ERR or OFF. The program will print
112 message on stdout and the default level is WARN.
113 • NM_CLOUD_SETUP_AZURE: boolean, whether Microsoft Azure support is
115 enabled. Defaults to no.
116 • NM_CLOUD_SETUP_EC2: boolean, whether Amazon EC2 (AWS) support is
118 enabled. Defaults to no.
119 • NM_CLOUD_SETUP_GCP: boolean, whether Google GCP support is enabled.
121 Defaults to no.
122
124 Amazon EC2 (AWS)
125 For AWS, the tools tries to fetch configuration from
126 http://169.254.169.254/. Currently, it only configures IPv4 and does
127 nothing about IPv6. It will do the following.
128 • First fetch http://169.254.169.254/latest/meta-data/ to determine
130 whether the expected API is present. This determines whether EC2
131 environment is detected and whether to proceed to configure the
132 host using EC2 meta data.
133 • Fetch
135 http://169.254.169.254/2018-09-24/meta-data/network/interfaces/macs/
136 to get the list of available interface. Interfaces are identified
137 by their MAC address.
138 • Then for each interface fetch
140 http://169.254.169.254/2018-09-24/meta-data/network/interfaces/macs/$MAC/subnet-ipv4-cidr-block
141 and
142 http://169.254.169.254/2018-09-24/meta-data/network/interfaces/macs/$MAC/local-ipv4s.
143 Thereby we get a list of local IPv4 addresses and one CIDR subnet
144 block.
145 • Then nm-cloud-setup iterates over all interfaces for which it could
147 fetch IP configuration. If no ethernet device for the respective
148 MAC address is found, it is skipped. Also, if the device is
149 currently not activated in NetworkManager or if the currently
150 activated profile has a user-data
151 org.freedesktop.nm-cloud-setup.skip=yes, it is skipped.
152 Then, the tool will change the runtime configuration of the device.
154 • Add static IPv4 addresses for all the configured addresses from
156 local-ipv4s with prefix length according to
157 subnet-ipv4-cidr-block. For example, we might have here 2 IP
158 addresses like "172.16.5.3/24,172.16.5.4/24".
159 • Choose a route table 30400 + the index of the interface and add
161 a default route 0.0.0.0/0. The gateway is the first IP address
162 in the CIDR subnet block. For example, we might get a route
163 "0.0.0.0/0 172.16.5.1 10 table=30401".
164 • Finally, add a policy routing rule for each address. For
166 example "priority 30401 from 172.16.5.3/32 table 30401,
167 priority 30401 from 172.16.5.4/32 table 30401".
168 With above example, this roughly corresponds for interface eth0 to
170 nmcli device modify "eth0" ipv4.addresses
171 "172.16.5.3/24,172.16.5.4/24" ipv4.routes "0.0.0.0/0 172.16.5.1 10
172 table=30401" ipv4.routing-rules "priority 30401 from 172.16.5.3/32
173 table 30401, priority 30401 from 172.16.5.4/32 table 30401". Note
174 that this replaces the previous addresses, routes and rules with
175 the new information. But also note that this only changes the run
176 time configuration of the device. The connection profile on disk is
177 not affected.
178 Google Cloud Platform (GCP)
180 For GCP, the meta data is fetched from URIs starting with
181 http://metadata.google.internal/computeMetadata/v1/ with a HTTP header
182 "Metadata-Flavor: Google". Currently, the tool only configures IPv4 and
183 does nothing about IPv6. It will do the following.
184 • First fetch
186 http://metadata.google.internal/computeMetadata/v1/instance/id to
187 detect whether the tool runs on Google Cloud Platform. Only if the
188 platform is detected, it will continue fetching the configuration.
189 • Fetch
191 http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/
192 to get the list of available interface indexes. These indexes can
193 be used for further lookups.
194 • Then, for each interface fetch
196 http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/$IFACE_INDEX/mac
197 to get the corresponding MAC address of the found interfaces. The
198 MAC address is used to identify the device later on.
199 • Then, for each interface with a MAC address fetch
201 http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/$IFACE_INDEX/forwarded-ips/
202 and then all the found IP addresses at
203 http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/$IFACE_INDEX/forwarded-ips/$FIPS_INDEX.
204 • At this point, we have a list of all interfaces (by MAC address)
206 and their configured IPv4 addresses.
207 For each device, we lookup the currently applied connection in
209 NetworkManager. That implies, that the device is currently
210 activated in NetworkManager. If no such device was in
211 NetworkManager, or if the profile has user-data
212 org.freedesktop.nm-cloud-setup.skip=yes, we skip the device. Now
213 for each found IP address we add a static route "$FIPS_ADDR/32
214 0.0.0.0 100 type=local" and reapply the change.
215 The effect is not unlike calling nmcli device modify "$DEVICE"
217 ipv4.routes "$FIPS_ADDR/32 0.0.0.0 100 type=local [,...]" for all
218 relevant devices and all found addresses.
219 Microsoft Azure
221 For Azure, the meta data is fetched from URIs starting with
222 http://169.254.169.254/metadata/instance with a URL parameter
223 "?format=text&api-version=2017-04-02" and a HTTP header
224 "Metadata:true". Currently, the tool only configures IPv4 and does
225 nothing about IPv6. It will do the following.
226 • First fetch
228 http://169.254.169.254/metadata/instance?format=text&api-version=2017-04-02
229 to detect whether the tool runs on Azure Cloud. Only if the
230 platform is detected, it will continue fetching the configuration.
231 • Fetch
233 http://169.254.169.254/metadata/instance/network/interface/?format=text&api-version=2017-04-02
234 to get the list of available interface indexes. These indexes can
235 be used for further lookups.
236 • Then, for each interface fetch
238 http://169.254.169.254/metadata/instance/network/interface/$IFACE_INDEX/macAddress?format=text&api-version=2017-04-02
239 to get the corresponding MAC address of the found interfaces. The
240 MAC address is used to identify the device later on.
241 • Then, for each interface with a MAC address fetch
243 http://169.254.169.254/metadata/instance/network/interface/$IFACE_INDEX/ipv4/ipAddress/?format=text&api-version=2017-04-02
244 to get the list of (indexes of) IP addresses on that interface.
245 • Then, for each IP address index fetch the address at
247 http://169.254.169.254/metadata/instance/network/interface/$IFACE_INDEX/ipv4/ipAddress/$ADDR_INDEX/privateIpAddress?format=text&api-version=2017-04-02.
248 Also fetch the size of the subnet (the netmask) for the interface
249 from
250 http://169.254.169.254/metadata/instance/network/interface/$IFACE_INDEX/ipv4/subnet/0/prefix/?format=text&api-version=2017-04-02.
251 • At this point, we have a list of all interfaces (by MAC address)
253 and their configured IPv4 addresses.
254 For each device, we lookup the currently applied connection in
256 NetworkManager. That implies, that the device is currently
257 activated in NetworkManager. If no such device was in
258 NetworkManager, or if the profile has user-data
259 org.freedesktop.nm-cloud-setup.skip=yes, we skip the device. Now
260 for each found IP address we add a static address
261 "$ADDR/$SUBNET_PREFIX". Also we configure policy routing by adding
262 a static route "$ADDR/$SUBNET_PREFIX $GATEWAY 10, table=$TABLE"
263 where $GATEWAY is the first IP address in the subnet and table is
264 30400 plus the interface index. Also we add a policy routing rule
265 "priority $TABLE from $ADDR/32 table $TABLE".
266 The effect is not unlike calling nmcli device modify "$DEVICE"
268 ipv4.addresses "$ADDR/$SUBNET [,...]" ipv4.routes "$ADDR/32
269 $GATEWAY 10 table=$TABLE" ipv4.routing-rules "priority $TABLE from
270 $ADDR/32 table $TABLE" for all relevant devices and all found
271 addresses.
272
274 NetworkManager(8) nmcli(1)
275NetworkManager 1.30.4 NM-CLOUD-SETUP(8)