1NM-CLOUD-SETUP(8)       Automatic Network Configuratio       NM-CLOUD-SETUP(8)
2
3
4

NAME

6       nm-cloud-setup - Overview of Automatic Network Configuration in Cloud
7

OVERVIEW

9       When running a virtual machine in a public cloud environment, it is
10       desirable to automatically configure the network of that VM. In simple
11       setups, the VM only has one network interface and the public cloud
12       supports automatic configuration via DHCP, DHCP6 or IPv6 autoconf.
13       However, the virtual machine might have multiple network interfaces, or
14       multiple IP addresses and IP subnets on one interface which cannot be
15       configured via DHCP. Also, the administrator may reconfigure the
16       network while the machine is running. NetworkManager's nm-cloud-setup
17       is a tool that automatically picks up such configuration in cloud
18       environments and updates the network configuration of the host.
19
20       Multiple cloud providers are supported. See the section called
21       “SUPPORTED CLOUD PROVIDERS”.
22

USE

24       The goal of nm-cloud-setup is to be configuration-less and work
25       automatically. All you need is to opt-in to the desired cloud providers
26       (see the section called “ENVIRONMENT VARIABLES”) and run
27       /usr/libexec/nm-cloud-setup.
28
29       Usually this is done by enabling the nm-cloud-setup.service systemd
30       service and let it run periodically. For that there is both a
31       nm-cloud-setup.timer systemd timer and a NetworkManager dispatcher
32       script.
33

DETAILS

35       nm-cloud-setup configures the network by fetching the configuration
36       from the well-known meta data server of the cloud provider. That means,
37       it already needs the network configured to the point where it can reach
38       the meta data server. Commonly that means, that a simple connection
39       profile is activated that possibly uses DHCP to get the primary IP
40       address. NetworkManager will create such a profile for ethernet devices
41       automatically if it is not configured otherwise via "no-auto-default"
42       setting in NetworkManager.conf. One possible alternative may be to
43       create such an initial profile with nmcli device connect "$DEVICE" or
44       nmcli connection add type ethernet ....
45
46       By setting the user-data org.freedesktop.nm-cloud-setup.skip=yes on the
47       profile, nm-cloud-setup will skip the device.
48
49       nm-cloud-setup modifies the run time configuration akin to nmcli device
50       modify. With this approach, the configuration is not persisted and only
51       preserved until the device disconnects.
52
53   /usr/libexec/nm-cloud-setup
54       The binary /usr/libexec/nm-cloud-setup does most of the work. It
55       supports no command line arguments but can be configured via
56       environment variables. See the section called “ENVIRONMENT VARIABLES”
57       for the supported environment variables.
58
59       By default, all cloud providers are disabled unless you opt-in by
60       enabling one or several providers. If cloud providers are enabled, the
61       program tries to fetch the host's configuration from a meta data server
62       of the cloud via HTTP. If configuration could be not fetched, no cloud
63       provider are detected and the program quits. If host configuration is
64       obtained, the corresponding cloud provider is successfully detected.
65       Then the network of the host will be configured.
66
67       It is intended to re-run nm-cloud-setup every time when the
68       configuration (maybe) changes. The tool is idempotent, so it should be
69       OK to also run it more often than necessary. You could run
70       /usr/libexec/nm-cloud-setup directly. However it may be preferable to
71       restart the nm-cloud-setup systemd service instead or use the timer or
72       dispatcher script to run it periodically (see below).
73
74   nm-cloud-setup.service systemd unit
75       Usually /usr/libexec/nm-cloud-setup is not run directly, but only by
76       systemctl restart nm-cloud-setup.service. This ensures that the tool
77       only runs once at any time. It also allows to integrate with the
78       nm-cloud-setup systemd timer, and to enable/disable the service via
79       systemd.
80
81       As you need to set environment variable to configure nm-cloud-setup
82       binary, you can do so via systemd override files. Try systemctl edit
83       nm-cloud-setup.service.
84
85   nm-cloud-setup.timer systemd timer
86       /usr/libexec/nm-cloud-setup is intended to run whenever an update is
87       necessary. For example, during boot when when changing the network
88       configuration of the virtual machine via the cloud provider.
89
90       One way to do this, is by enabling the nm-cloud-setup.timer systemd
91       timer with systemctl enable --now nm-cloud-setup.timer.
92
93   /usr/lib/NetworkManager/dispatcher.d/90-nm-cloud-setup.sh
94       There is also a NetworkManager dispatcher script that will run for
95       example when an interface is activated by NetworkManager. Together with
96       the nm-cloud-setup.timer systemd timer this script is to automatically
97       pick up changes to the network.
98
99       The dispatcher script will do nothing, unless the systemd service is
100       enabled. To use the dispatcher script you should therefor run systemctl
101       enable nm-cloud-setup.service once.
102

ENVIRONMENT VARIABLES

104       The following environment variables are used to configure
105       /usr/libexec/nm-cloud-setup. You may want to configure them with a
106       drop-in for the systemd service. For example by calling systemctl edit
107       nm-cloud-setup.service and configuring [Service] Environment=, as
108       described in systemd.exec(5) manual.
109
110       •   NM_CLOUD_SETUP_LOG: control the logging verbosity. Set it to one of
111           TRACE, DEBUG, INFO, WARN, ERR or OFF. The program will print
112           message on stdout and the default level is WARN.
113
114       •   NM_CLOUD_SETUP_AZURE: boolean, whether Microsoft Azure support is
115           enabled. Defaults to no.
116
117       •   NM_CLOUD_SETUP_EC2: boolean, whether Amazon EC2 (AWS) support is
118           enabled. Defaults to no.
119
120       •   NM_CLOUD_SETUP_GCP: boolean, whether Google GCP support is enabled.
121           Defaults to no.
122
123       •   NM_CLOUD_SETUP_ALIYUN: boolean, whether Alibaba Cloud (Aliyun)
124           support is enabled. Defaults to no.
125

EXAMPLE SETUP FOR CONFIGURING AND PREDEPLOYING NM-CLOUD-SETUP

127       As detailed before, nm-cloud-setup needs to be explicitly enabled. As
128       it runs as a systemd service and timer, that basically means to enable
129       and configure those. This can be done by dropping the correct files and
130       symlinks to disk.
131
132       The following example enables nm-cloud-setup for Amazon EC2 cloud:
133
134           dnf install -y NetworkManager-cloud-setup
135
136           mkdir -p /etc/systemd/system/nm-cloud-setup.service.d
137           cat > /etc/systemd/system/nm-cloud-setup.service.d/10-enable-ec2.conf << EOF
138           [Service]
139           Environment=NM_CLOUD_SETUP_EC2=yes
140           EOF
141
142           # systemctl enable nm-cloud-setup.service
143           mkdir -p /etc/systemd/system/NetworkManager.service.wants/
144           ln -s /usr/lib/systemd/system/nm-cloud-setup.service /etc/systemd/system/NetworkManager.service.wants/nm-cloud-setup.service
145
146           # systemctl enable nm-cloud-setup.timer
147           mkdir -p /etc/systemd/system/timers.target.wants/
148           ln -s /etc/systemd/system/timers.target.wants/nm-cloud-setup.timer /usr/lib/systemd/system/nm-cloud-setup.timer
149
150           # systemctl daemon-reload
151
152
153

SUPPORTED CLOUD PROVIDERS

155   Amazon EC2 (AWS)
156       For AWS, the tools tries to fetch configuration from
157       http://169.254.169.254/. Currently, it only configures IPv4 and does
158       nothing about IPv6. It will do the following.
159
160       •   First fetch http://169.254.169.254/latest/meta-data/ to determine
161           whether the expected API is present. This determines whether EC2
162           environment is detected and whether to proceed to configure the
163           host using EC2 meta data.
164
165       •   Fetch
166           http://169.254.169.254/2018-09-24/meta-data/network/interfaces/macs/
167           to get the list of available interface. Interfaces are identified
168           by their MAC address.
169
170       •   Then for each interface fetch
171           http://169.254.169.254/2018-09-24/meta-data/network/interfaces/macs/$MAC/subnet-ipv4-cidr-block
172           and
173           http://169.254.169.254/2018-09-24/meta-data/network/interfaces/macs/$MAC/local-ipv4s.
174           Thereby we get a list of local IPv4 addresses and one CIDR subnet
175           block.
176
177       •   Then nm-cloud-setup iterates over all interfaces for which it could
178           fetch IP configuration. If no ethernet device for the respective
179           MAC address is found, it is skipped. Also, if the device is
180           currently not activated in NetworkManager or if the currently
181           activated profile has a user-data
182           org.freedesktop.nm-cloud-setup.skip=yes, it is skipped.
183
184           If only one interface and one address is configured, then the tool
185           does nothing and leaves the automatic configuration that was
186           obtained via DHCP.
187
188           Otherwise, the tool will change the runtime configuration of the
189           device.
190
191           •   Add static IPv4 addresses for all the configured addresses from
192               local-ipv4s with prefix length according to
193               subnet-ipv4-cidr-block. For example, we might have here 2 IP
194               addresses like "172.16.5.3/24,172.16.5.4/24".
195
196           •   Choose a route table 30400 + the index of the interface and add
197               a default route 0.0.0.0/0. The gateway is the first IP address
198               in the CIDR subnet block. For example, we might get a route
199               "0.0.0.0/0 172.16.5.1 10 table=30400".
200
201               Also choose a route table 30200 + the interface index. This
202               contains a direct routes to the subnets of this interface.
203
204           •   Finally, add a policy routing rule for each address. For
205               example "priority 30200 from 172.16.5.3/32 table 30200,
206               priority 30200 from 172.16.5.4/32 table 30200". and "priority
207               30400 from 172.16.5.3/32 table 30400, priority 30400 from
208               172.16.5.4/32 table 30400" The 30200+ rules select the table to
209               reach the subnet directly, while the 30400+ rules use the
210               default route. Also add a rule "priority 30350 table main
211               suppress_prefixlength 0". This has a priority between the two
212               previous rules and causes a lookup of routes in the main table
213               while ignoring the default route. The purpose of this is so
214               that other specific routes in the main table are honored over
215               the default route in table 30400+.
216
217           With above example, this roughly corresponds for interface eth0 to
218           nmcli device modify "eth0" ipv4.addresses
219           "172.16.5.3/24,172.16.5.4/24" ipv4.routes "172.16.5.0/24 0.0.0.0 10
220           table=30200, 0.0.0.0/0 172.16.5.1 10 table=30400"
221           ipv4.routing-rules "priority 30200 from 172.16.5.3/32 table 30200,
222           priority 30200 from 172.16.5.4/32 table 30200, priority 20350 table
223           main suppress_prefixlength 0, priority 30400 from 172.16.5.3/32
224           table 30400, priority 30400 from 172.16.5.4/32 table 30400". Note
225           that this replaces the previous addresses, routes and rules with
226           the new information. But also note that this only changes the run
227           time configuration of the device. The connection profile on disk is
228           not affected.
229
230   Google Cloud Platform (GCP)
231       For GCP, the meta data is fetched from URIs starting with
232       http://metadata.google.internal/computeMetadata/v1/ with a HTTP header
233       "Metadata-Flavor: Google". Currently, the tool only configures IPv4 and
234       does nothing about IPv6. It will do the following.
235
236       •   First fetch
237           http://metadata.google.internal/computeMetadata/v1/instance/id to
238           detect whether the tool runs on Google Cloud Platform. Only if the
239           platform is detected, it will continue fetching the configuration.
240
241       •   Fetch
242           http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/
243           to get the list of available interface indexes. These indexes can
244           be used for further lookups.
245
246       •   Then, for each interface fetch
247           http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/$IFACE_INDEX/mac
248           to get the corresponding MAC address of the found interfaces. The
249           MAC address is used to identify the device later on.
250
251       •   Then, for each interface with a MAC address fetch
252           http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/$IFACE_INDEX/forwarded-ips/
253           and then all the found IP addresses at
254           http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/$IFACE_INDEX/forwarded-ips/$FIPS_INDEX.
255
256       •   At this point, we have a list of all interfaces (by MAC address)
257           and their configured IPv4 addresses.
258
259           For each device, we lookup the currently applied connection in
260           NetworkManager. That implies, that the device is currently
261           activated in NetworkManager. If no such device was in
262           NetworkManager, or if the profile has user-data
263           org.freedesktop.nm-cloud-setup.skip=yes, we skip the device. Now
264           for each found IP address we add a static route "$FIPS_ADDR/32
265           0.0.0.0 100 type=local" and reapply the change.
266
267           The effect is not unlike calling nmcli device modify "$DEVICE"
268           ipv4.routes "$FIPS_ADDR/32 0.0.0.0 100 type=local [,...]"  for all
269           relevant devices and all found addresses.
270
271   Microsoft Azure
272       For Azure, the meta data is fetched from URIs starting with
273       http://169.254.169.254/metadata/instance with a URL parameter
274       "?format=text&api-version=2017-04-02" and a HTTP header
275       "Metadata:true". Currently, the tool only configures IPv4 and does
276       nothing about IPv6. It will do the following.
277
278       •   First fetch
279           http://169.254.169.254/metadata/instance?format=text&api-version=2017-04-02
280           to detect whether the tool runs on Azure Cloud. Only if the
281           platform is detected, it will continue fetching the configuration.
282
283       •   Fetch
284           http://169.254.169.254/metadata/instance/network/interface/?format=text&api-version=2017-04-02
285           to get the list of available interface indexes. These indexes can
286           be used for further lookups.
287
288       •   Then, for each interface fetch
289           http://169.254.169.254/metadata/instance/network/interface/$IFACE_INDEX/macAddress?format=text&api-version=2017-04-02
290           to get the corresponding MAC address of the found interfaces. The
291           MAC address is used to identify the device later on.
292
293       •   Then, for each interface with a MAC address fetch
294           http://169.254.169.254/metadata/instance/network/interface/$IFACE_INDEX/ipv4/ipAddress/?format=text&api-version=2017-04-02
295           to get the list of (indexes of) IP addresses on that interface.
296
297       •   Then, for each IP address index fetch the address at
298           http://169.254.169.254/metadata/instance/network/interface/$IFACE_INDEX/ipv4/ipAddress/$ADDR_INDEX/privateIpAddress?format=text&api-version=2017-04-02.
299           Also fetch the size of the subnet and prefix for the interface from
300           http://169.254.169.254/metadata/instance/network/interface/$IFACE_INDEX/ipv4/subnet/0/address/?format=text&api-version=2017-04-02.
301           and
302           http://169.254.169.254/metadata/instance/network/interface/$IFACE_INDEX/ipv4/subnet/0/prefix/?format=text&api-version=2017-04-02.
303
304       •   At this point, we have a list of all interfaces (by MAC address)
305           and their configured IPv4 addresses.
306
307           Then the tool configures the system like doing for AWS environment.
308           That is, using source based policy routing with the tables/rules
309           30200/30400.
310
311   Alibaba Cloud (Aliyun)
312       For Aliyun, the tools tries to fetch configuration from
313       http://100.100.100.200/. Currently, it only configures IPv4 and does
314       nothing about IPv6. It will do the following.
315
316       •   First fetch http://100.100.100.200/2016-01-01/meta-data/ to
317           determine whether the expected API is present. This determines
318           whether Aliyun environment is detected and whether to proceed to
319           configure the host using Aliyun meta data.
320
321       •   Fetch
322           http://100.100.100.200/2016-01-01/meta-data/network/interfaces/macs/
323           to get the list of available interface. Interfaces are identified
324           by their MAC address.
325
326       •   Then for each interface fetch
327           http://100.100.100.200/2016-01-01/meta-data/network/interfaces/macs/$MAC/vpc-cidr-block,
328           http://100.100.100.200/2016-01-01/meta-data/network/interfaces/macs/$MAC/private-ipv4s,
329           http://100.100.100.200/2016-01-01/meta-data/network/interfaces/macs/$MAC/netmask
330           and
331           http://100.100.100.200/2016-01-01/meta-data/network/interfaces/macs/$MAC/gateway.
332           Thereby we get a list of private IPv4 addresses, one CIDR subnet
333           block and private IPv4 addresses prefix.
334
335       •   Then nm-cloud-setup iterates over all interfaces for which it could
336           fetch IP configuration. If no ethernet device for the respective
337           MAC address is found, it is skipped. Also, if the device is
338           currently not activated in NetworkManager or if the currently
339           activated profile has a user-data
340           org.freedesktop.nm-cloud-setup.skip=yes, it is skipped. Also, there
341           is only one interface and one IP address, the tool does nothing.
342
343           Then the tool configures the system like doing for AWS environment.
344           That is, using source based policy routing with the tables/rules
345           30200/30400. One difference to AWS is that the gateway is also
346           fetched via metadata instead of using the first IP address in the
347           subnet.
348

SEE ALSO

350       NetworkManager(8) nmcli(1)
351
352
353
354NetworkManager 1.38.0                                        NM-CLOUD-SETUP(8)
Impressum