1NM-CLOUD-SETUP(8)       Automatic Network Configuratio       NM-CLOUD-SETUP(8)
2
3
4

NAME

6       nm-cloud-setup - Overview of Automatic Network Configuration in Cloud
7

OVERVIEW

9       When running a virtual machine in a public cloud environment, it is
10       desirable to automatically configure the network of that VM. In simple
11       setups, the VM only has one network interface and the public cloud
12       supports automatic configuration via DHCP, DHCP6 or IPv6 autoconf.
13       However, the virtual machine might have multiple network interfaces, or
14       multiple IP addresses and IP subnets on one interface which cannot be
15       configured via DHCP. Also, the administrator may reconfigure the
16       network while the machine is running. NetworkManager's nm-cloud-setup
17       is a tool that automatically picks up such configuration in cloud
18       environments and updates the network configuration of the host.
19
20       Multiple cloud providers are supported. See the section called
21       “SUPPORTED CLOUD PROVIDERS”.
22

USE

24       The goal of nm-cloud-setup is to be configuration-less and work
25       automatically. All you need is to opt-in to the desired cloud providers
26       (see the section called “ENVIRONMENT VARIABLES”) and run
27       /usr/libexec/nm-cloud-setup.
28
29       Usually this is done by enabling the nm-cloud-setup.service systemd
30       service and let it run periodically. For that there is both a
31       nm-cloud-setup.timer systemd timer and a NetworkManager dispatcher
32       script.
33

DETAILS

35       nm-cloud-setup configures the network by fetching the configuration
36       from the well-known meta data server of the cloud provider. That means,
37       it already needs the network configured to the point where it can reach
38       the meta data server. Commonly that means, that a simple connection
39       profile is activated that possibly uses DHCP to get the primary IP
40       address. NetworkManager will create such a profile for ethernet devices
41       automatically if it is not configured otherwise via "no-auto-default"
42       setting in NetworkManager.conf. One possible alternative may be to
43       create such an initial profile with nmcli device connect "$DEVICE" or
44       nmcli connection add type ethernet ....
45
46       By setting the user-data org.freedesktop.nm-cloud-setup.skip=yes on the
47       profile, nm-cloud-setup will skip the device.
48
49       nm-cloud-setup modifies the run time configuration akin to nmcli device
50       modify. With this approach, the configuration is not persisted and only
51       preserved until the device disconnects.
52
53   /usr/libexec/nm-cloud-setup
54       The binary /usr/libexec/nm-cloud-setup does most of the work. It
55       supports no command line arguments but can be configured via
56       environment variables. See the section called “ENVIRONMENT VARIABLES”
57       for the supported environment variables.
58
59       By default, all cloud providers are disabled unless you opt-in by
60       enabling one or several providers. If cloud providers are enabled, the
61       program tries to fetch the host's configuration from a meta data server
62       of the cloud via HTTP. If configuration could be not fetched, no cloud
63       provider are detected and the program quits. If host configuration is
64       obtained, the corresponding cloud provider is successfully detected.
65       Then the network of the host will be configured.
66
67       It is intended to re-run nm-cloud-setup every time when the
68       configuration (maybe) changes. The tool is idempotent, so it should be
69       OK to also run it more often than necessary. You could run
70       /usr/libexec/nm-cloud-setup directly. However it may be preferable to
71       restart the nm-cloud-setup systemd service instead or use the timer or
72       dispatcher script to run it periodically (see below).
73
74   nm-cloud-setup.service systemd unit
75       Usually /usr/libexec/nm-cloud-setup is not run directly, but only by
76       systemctl restart nm-cloud-setup.service. This ensures that the tool
77       only runs once at any time. It also allows to integrate with the
78       nm-cloud-setup systemd timer, and to enable/disable the service via
79       systemd.
80
81       As you need to set environment variable to configure nm-cloud-setup
82       binary, you can do so via systemd override files. Try systemctl edit
83       nm-cloud-setup.service.
84
85   nm-cloud-setup.timer systemd timer
86       /usr/libexec/nm-cloud-setup is intended to run whenever an update is
87       necessary. For example, during boot when when changing the network
88       configuration of the virtual machine via the cloud provider.
89
90       One way to do this, is by enabling the nm-cloud-setup.timer systemd
91       timer with systemctl enable --now nm-cloud-setup.timer.
92
93   /usr/lib/NetworkManager/dispatcher.d/90-nm-cloud-setup.sh
94       There is also a NetworkManager dispatcher script that will run for
95       example when an interface is activated by NetworkManager. Together with
96       the nm-cloud-setup.timer systemd timer this script is to automatically
97       pick up changes to the network.
98
99       The dispatcher script will do nothing, unless the systemd service is
100       enabled. To use the dispatcher script you should therefor run systemctl
101       enable nm-cloud-setup.service once.
102

ENVIRONMENT VARIABLES

104       The following environment variables are used to configure
105       /usr/libexec/nm-cloud-setup. You may want to configure them with a
106       drop-in for the systemd service. For example by calling systemctl edit
107       nm-cloud-setup.service and configuring [Service] Environment=, as
108       described in systemd.exec(5) manual.
109
110       •   NM_CLOUD_SETUP_LOG: control the logging verbosity. Set it to one of
111           TRACE, DEBUG, INFO, WARN, ERR or OFF. The program will print
112           message on stdout and the default level is WARN.
113
114       •   NM_CLOUD_SETUP_AZURE: boolean, whether Microsoft Azure support is
115           enabled. Defaults to no.
116
117       •   NM_CLOUD_SETUP_EC2: boolean, whether Amazon EC2 (AWS) support is
118           enabled. Defaults to no.
119
120       •   NM_CLOUD_SETUP_GCP: boolean, whether Google GCP support is enabled.
121           Defaults to no.
122

SUPPORTED CLOUD PROVIDERS

124   Amazon EC2 (AWS)
125       For AWS, the tools tries to fetch configuration from
126       http://169.254.169.254/. Currently, it only configures IPv4 and does
127       nothing about IPv6. It will do the following.
128
129       •   First fetch http://169.254.169.254/latest/meta-data/ to determine
130           whether the expected API is present. This determines whether EC2
131           environment is detected and whether to proceed to configure the
132           host using EC2 meta data.
133
134       •   Fetch
135           http://169.254.169.254/2018-09-24/meta-data/network/interfaces/macs/
136           to get the list of available interface. Interfaces are identified
137           by their MAC address.
138
139       •   Then for each interface fetch
140           http://169.254.169.254/2018-09-24/meta-data/network/interfaces/macs/$MAC/subnet-ipv4-cidr-block
141           and
142           http://169.254.169.254/2018-09-24/meta-data/network/interfaces/macs/$MAC/local-ipv4s.
143           Thereby we get a list of local IPv4 addresses and one CIDR subnet
144           block.
145
146       •   Then nm-cloud-setup iterates over all interfaces for which it could
147           fetch IP configuration. If no ethernet device for the respective
148           MAC address is found, it is skipped. Also, if the device is
149           currently not activated in NetworkManager or if the currently
150           activated profile has a user-data
151           org.freedesktop.nm-cloud-setup.skip=yes, it is skipped.
152
153           Then, the tool will change the runtime configuration of the device.
154
155           •   Add static IPv4 addresses for all the configured addresses from
156               local-ipv4s with prefix length according to
157               subnet-ipv4-cidr-block. For example, we might have here 2 IP
158               addresses like "172.16.5.3/24,172.16.5.4/24".
159
160           •   Choose a route table 30400 + the index of the interface and add
161               a default route 0.0.0.0/0. The gateway is the first IP address
162               in the CIDR subnet block. For example, we might get a route
163               "0.0.0.0/0 172.16.5.1 10 table=30401".
164
165           •   Finally, add a policy routing rule for each address. For
166               example "priority 30401 from 172.16.5.3/32 table 30401,
167               priority 30401 from 172.16.5.4/32 table 30401".
168
169           With above example, this roughly corresponds for interface eth0 to
170           nmcli device modify "eth0" ipv4.addresses
171           "172.16.5.3/24,172.16.5.4/24" ipv4.routes "0.0.0.0/0 172.16.5.1 10
172           table=30401" ipv4.routing-rules "priority 30401 from 172.16.5.3/32
173           table 30401, priority 30401 from 172.16.5.4/32 table 30401". Note
174           that this replaces the previous addresses, routes and rules with
175           the new information. But also note that this only changes the run
176           time configuration of the device. The connection profile on disk is
177           not affected.
178
179   Google Cloud Platform (GCP)
180       For GCP, the meta data is fetched from URIs starting with
181       http://metadata.google.internal/computeMetadata/v1/ with a HTTP header
182       "Metadata-Flavor: Google". Currently, the tool only configures IPv4 and
183       does nothing about IPv6. It will do the following.
184
185       •   First fetch
186           http://metadata.google.internal/computeMetadata/v1/instance/id to
187           detect whether the tool runs on Google Cloud Platform. Only if the
188           platform is detected, it will continue fetching the configuration.
189
190       •   Fetch
191           http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/
192           to get the list of available interface indexes. These indexes can
193           be used for further lookups.
194
195       •   Then, for each interface fetch
196           http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/$IFACE_INDEX/mac
197           to get the corresponding MAC address of the found interfaces. The
198           MAC address is used to identify the device later on.
199
200       •   Then, for each interface with a MAC address fetch
201           http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/$IFACE_INDEX/forwarded-ips/
202           and then all the found IP addresses at
203           http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/$IFACE_INDEX/forwarded-ips/$FIPS_INDEX.
204
205       •   At this point, we have a list of all interfaces (by MAC address)
206           and their configured IPv4 addresses.
207
208           For each device, we lookup the currently applied connection in
209           NetworkManager. That implies, that the device is currently
210           activated in NetworkManager. If no such device was in
211           NetworkManager, or if the profile has user-data
212           org.freedesktop.nm-cloud-setup.skip=yes, we skip the device. Now
213           for each found IP address we add a static route "$FIPS_ADDR/32
214           0.0.0.0 100 type=local" and reapply the change.
215
216           The effect is not unlike calling nmcli device modify "$DEVICE"
217           ipv4.routes "$FIPS_ADDR/32 0.0.0.0 100 type=local [,...]"  for all
218           relevant devices and all found addresses.
219
220   Microsoft Azure
221       For Azure, the meta data is fetched from URIs starting with
222       http://169.254.169.254/metadata/instance with a URL parameter
223       "?format=text&api-version=2017-04-02" and a HTTP header
224       "Metadata:true". Currently, the tool only configures IPv4 and does
225       nothing about IPv6. It will do the following.
226
227       •   First fetch
228           http://169.254.169.254/metadata/instance?format=text&api-version=2017-04-02
229           to detect whether the tool runs on Azure Cloud. Only if the
230           platform is detected, it will continue fetching the configuration.
231
232       •   Fetch
233           http://169.254.169.254/metadata/instance/network/interface/?format=text&api-version=2017-04-02
234           to get the list of available interface indexes. These indexes can
235           be used for further lookups.
236
237       •   Then, for each interface fetch
238           http://169.254.169.254/metadata/instance/network/interface/$IFACE_INDEX/macAddress?format=text&api-version=2017-04-02
239           to get the corresponding MAC address of the found interfaces. The
240           MAC address is used to identify the device later on.
241
242       •   Then, for each interface with a MAC address fetch
243           http://169.254.169.254/metadata/instance/network/interface/$IFACE_INDEX/ipv4/ipAddress/?format=text&api-version=2017-04-02
244           to get the list of (indexes of) IP addresses on that interface.
245
246       •   Then, for each IP address index fetch the address at
247           http://169.254.169.254/metadata/instance/network/interface/$IFACE_INDEX/ipv4/ipAddress/$ADDR_INDEX/privateIpAddress?format=text&api-version=2017-04-02.
248           Also fetch the size of the subnet (the netmask) for the interface
249           from
250           http://169.254.169.254/metadata/instance/network/interface/$IFACE_INDEX/ipv4/subnet/0/prefix/?format=text&api-version=2017-04-02.
251
252       •   At this point, we have a list of all interfaces (by MAC address)
253           and their configured IPv4 addresses.
254
255           For each device, we lookup the currently applied connection in
256           NetworkManager. That implies, that the device is currently
257           activated in NetworkManager. If no such device was in
258           NetworkManager, or if the profile has user-data
259           org.freedesktop.nm-cloud-setup.skip=yes, we skip the device. Now
260           for each found IP address we add a static address
261           "$ADDR/$SUBNET_PREFIX". Also we configure policy routing by adding
262           a static route "$ADDR/$SUBNET_PREFIX $GATEWAY 10, table=$TABLE"
263           where $GATEWAY is the first IP address in the subnet and table is
264           30400 plus the interface index. Also we add a policy routing rule
265           "priority $TABLE from $ADDR/32 table $TABLE".
266
267           The effect is not unlike calling nmcli device modify "$DEVICE"
268           ipv4.addresses "$ADDR/$SUBNET [,...]" ipv4.routes "$ADDR/32
269           $GATEWAY 10 table=$TABLE" ipv4.routing-rules "priority $TABLE from
270           $ADDR/32 table $TABLE" for all relevant devices and all found
271           addresses.
272

SEE ALSO

274       NetworkManager(8) nmcli(1)
275
276
277
278NetworkManager 1.30.4                                        NM-CLOUD-SETUP(8)
Impressum