1VM::EC2::Staging::VolumUes(e3rpmC)ontributed Perl DocumeVnMt:a:tEiCo2n::Staging::Volume(3pm)
2
6 VM::EC2::Staging::Volume - High level functions for provisioning and
7 populating EC2 volumes
8
10 use VM::EC2::Staging::manager;
11 # get a new staging manager
13 my $ec2 = VM::EC2->new;
14 my $staging = $ec2->staging_manager(); );
15 my $vol1 = $staging->get_volume(-name => 'Backup',
17 -fstype => 'ext4',
18 -size => 11,
19 -zone => 'us-east-1a');
20 # make a couple of directories in new volume
22 $vol1->mkdir('pictures');
23 $vol1->mkdir('videos');
24 # use rsync to copy local files onto a subdirectory of this volume
26 $vol1->put('/usr/local/my_pictures/' =>'pictures');
27 $vol1->put('/usr/local/my_videos/' =>'videos');
28 # use rsync to to copy a set of files on the volume to a local directory
30 mkdir('/tmp/jpegs');
31 $vol1->get('pictures/*.jpg','/tmp/jpegs');
32 # note that these commands are executed on the remote server as root!
34 @listing = $vol1->ls('-r','pictures');
35 $vol1->chown('fred','pictures');
36 $vol1->chgrp('nobody','pictures');
37 $vol1->chmod('0700','pictures');
38 $vol1->rm('-rf','pictures/*');
39 $vol1->rmdir('pictures');
40 # get some information about the volume
42 my $mtpt = $vol->mtpt;
43 my $mtdev = $vol->mtdev;
44 my $mounted = $vol->mounted;
45 my $server = $vol->server;
46 # detach the volume
48 $vol->detach;
49 # delete the volume entirely
51 $vol->delete;
52
54 This is a high-level interface to EBS volumes which is used in
55 conjunction with VM::EC2::Staging::Manager and
56 VM::EC2::Staging::Server. It is intended to ease the process of
57 allocating and managing EBS volumes, and provides for completely
58 automated filesystem creation, directory management, and data transfer
59 to and from the volume.
60 You can use staging volumes without having to manually create and
62 manage the instances needed to manipulate the volumes. As needed, the
63 staging manager will create the server(s) needed to execute the desired
64 actions on the volumes.
65 Staging volumes are wrappers around VM::EC2::Volume, and have all the
67 methods associated with those objects. In addition to the standard EC2
68 volume characteristics, each staging volume in an EC2 region has a
69 symbolic name, which can be used to retrieve previously-created volumes
70 without remembering their volume ID. This symbolic name is stored in
71 the tag StagingName. Volumes also have a filesystem type (stored in the
72 tag StagingFsType). When a volume is mounted on a staging server, it
73 will also have a mount point on the file system, and a mounting device
74 (e.g. /dev/sdf1).
75
77 Staging volumes are created via a staging manager's get_volume() or
78 provision_volume() methods. See VM::EC2::Staging::Manager. One typical
79 invocation is:
80 my $ec2 = VM::EC2->new;
82 my $manager = $ec2->staging_manager(); );
83 my $vol = $manager->get_volume(-name => 'Backup',
84 -fstype => 'ext4',
85 -size => 5,
86 -zone => 'us-east-1a');
87 This will either retrieve an existing volume named "Backup", or, if
89 none exists, create a new one using the provided specification. Behind
90 the scenes, a staging server will be allocated to mount the volume. The
91 manager tries to conserve resources, and so will reuse a suitable
92 running staging server if one is available.
93 The other typical invocation is:
95 my $vol = $manager->provision_volume(-name => 'Backup',
97 -fstype => 'ext4',
98 -size => 5,
99 -zone => 'us-east-1a');
100 This forces creation of a new volume with the indicated
102 characteristics. If a volume of the same name already exists, this
103 method will die with a fatal error (to avoid this, either wrap in an
104 eval, or leave off the -name argument and let the manager pick a unique
105 name for you).
106
108 The methods in this section return status information about the staging
109 volume.
110 $name = $vol->name([$newname])
112 Get/set the symbolic name associated with this volume.
113 $mounted = $vol->mounted
115 Returns true if the volume is currently mounted on a server.
116 $type = $vol->fstype
118 Return the filesystem type requested at volume creation time.
119 $server = $vol->server
121 Get the server associated with this volume, if any.
122 $device = $vol->mtdev
124 Get the device that the volume is attached to, e.g. /dev/sdf1. If the
125 volume is not attached to a server, returns undef.
126 $device = $vol->mtpt
128 Get the mount point for this volume on the attached server. If the
129 volume is not mounted, returns undef.
130 $ebs_vol = $vol->ebs
132 Get the underlying EBS volume associated with the staging volume
133 object.
134 $manager = $vol->manager
136 Return the VM::EC2::Staging::Manager which manages this volume.
137 $string = $vol->fstab_line();
139 This method returns the line in /etc/fstab that would be necessary to
140 mount this volume on the server to which it is currently attached at
141 boot time. For example:
142 /dev/sdf1 /mnt/staging/Backups ext4 defaults,nobootwait 0 2
144 You can add this to the current server's fstab using the following code
146 fragment:
147 my $server = $vol->server;
149 my $fh = $server->scmd_write('sudo -s "cat >>/etc/fstab"');
150 print $fh $vol->fstab,"\n";
151 close $fh;
152 $type = $vol->get_fstype
154 Return the volume's actual filesystem type. This can be different from
155 the requested type if it was later altered by running mkfs on the
156 volume, or the contents of the disk were overwritten by a block-level
157 dd command. As a side effect, this method sets fstype() to the current
158 correct value.
159
161 The methods in this section control the state of the volume.
162 $snapshot = $vol->create_snapshot('description')
164 Create a VM::EC2::Snapshot of the volume with an optional description.
165 This differs from the VM::EC2::Volume method of the same name in that
166 it is aware of the mount state of the volume and will first try to
167 unmount it so that the snapshot is clean. After the snapshot is
168 started, the volume is remounted.
169 $snapshot = $vol->snapshot('description')
171 Identical to create_snapshot(), but the method name is shorter.
172 $vol->mount($server [,$mtpt])
174 $vol->mount()
175 Mount the volume on the indicated VM::EC2::Staging::Server, optionally
176 at a named mount point on the file system. If the volume is already
177 attached to a different server, it will be detached first. If any of
178 these step fails, the method will die with a fatal error.
179 When called with no arguments, the volume is automounted on a staging
181 server, creating or starting the server if necessary.
182 $vol->unmount()
184 Unmount the volume from wherever it is, but leave it attached to the
185 staging server. If the volume is not already mounted, nothing happens.
186 Note that it is possible for a volume to be mounted on a stopped
188 server, in which case the server will be started and the volume only
189 unmounted when it is up and running.
190 $vol->detach()
192 Unmount and detach the volume from its current server, if any.
193 Note that it is possible for a volume to be mounted on a stopped
195 server, in which case the server will be started and the volume only
196 unmounted when it is up and running.
197 $vol->delete()
199 Delete the volume entirely. If it is mounted and/or attached to a
200 server, it will be unmounted/detached first. If any steps fail, the
201 method will die with a fatal error.
202
204 The methods in this section operate on the contents of the volume. By
205 and large, they operate with root privileges on the server machine via
206 judicious use of sudo. Elevated permissions on the local machine (on
207 which the script is running) are not needed.
208 $vol->get($source_on_vol_1,$source_on_vol_2,...,$dest)
210 Invoke rsync() on the server to copy files & directories from the
211 indicated source locations on the staging volume to the destination.
212 Source paths can be relative paths, such as "media/photos/vacation", in
213 which case they are relative to the top level of the mounted volume, or
214 absolute paths, such as "/usr/local/media/photos/vacation", in which
215 case they are treated as absolute paths on the server on which the
216 volume is mounted.
217 The destination can be a path on the local machine, a host:/path on a
219 remote machine, a staging server and path in the form $server:/path, or
220 a staging volume and path in the form "$volume/path". See "Instance
221 Methods for Managing Staging Volumes" in VM::EC2::Staging::Manager for
222 more formats you can use.
223 As a special case, if you invoke get() with a single argument:
225 $vol->get('/tmp/foo')
227 Then the entire volume will be rsynced into the destination directory
229 /tmp/foo.
230 $vol->copy($source_on_vol_1,$source_on_vol_2,...,$dest)
232 This is an alias for get(). It is intended to make it easier to read
233 the intent of this command:
234 $source_volume->copy($destination_volume);
236 Which basically makes a copy of $source_volume onto
238 $destination_volume.
239 $vol->put($source1,$source2,$source3,...,$dest_on_volume)
241 Invoke rsync() on the server to copy files & directories from the
242 indicated source locations a destination located on the staging volume.
243 The rules for paths are the same as for the get() method and as
244 described in "Instance Methods for Managing Staging Volumes" in
245 VM::EC2::Staging::Manager .
246 As a special case, if you invoke put() with a single argument:
248 $vol->put('/tmp/foo')
250 Then the local directory /tmp/foo will be copied onto the top level of
252 the staging volume. To do something similar with multiple source
253 directories, use '/' or '.' as the destination:
254 $vol->put('/tmp/pictures','/tmp/audio' => '/');
256 $vol->dd($destination_volume)
258 The dd() method performs a block level copy of the volume's disk onto
259 the destination. The destination must be another staging volume.
260 $output = $vol->cmd($cmd,@args)
262 This method runs command $cmd on the server that is mounting the volume
263 using ssh. Before the command is run, the working directory is changed
264 to the top level of the volume's mount point. Any arguments, switches,
265 etc you wish to pass to the command can be provided as @args. The
266 output of the command is returned as a string in a scalar context, or
267 an array of lines in a list context.
268 Example:
270 @log = $volume->cmd('tar cvf /tmp/archive.tar .');
272 $result = $vol->ssh($cmd,@args)
274 This is similar to cmd(), except that the output of the command is sent
275 to STDOUT and the method returns true if the command executed
276 succcessfully on the remote machine. The cmd() and ssh() methods are
277 equivalent to backticks are system() respectively.
278 Example:
280 $volume->ssh('gzip /tmp/archive.tar') or die "couldn't compress archive";
282 $output = $vol->df(@args)
284 $output = $vol->ls(@args)
285 $success = $vol->mkdir(@args)
286 $success = $vol->chown(@args)
287 $success = $vol->chgrp(@args)
288 $success = $vol->chmod(@args)
289 $success = $vol->cp(@args)
290 $success = $vol->mv(@args)
291 $success = $vol->rm(@args)
292 $success = $vol->rmdir(@args)
293 Each of these methods performs the same function as the like-named
294 command-line function, after first changing the working directory to
295 the top level of the volume. They behave as shown in the pseudocode
296 below:
297 chdir $vol->mtpt;
299 sudo $method @args
300 The df() and ls() methods return the output of their corresponding
302 commands. In a scalar context each method returns a string
303 corresponding to the output of running the command on the server to
304 which the volume is attached. In a list context, the methods return one
305 element per line of output.
306 For example:
308 my $free = $volume->df('.'); # free on current directory
310 my ($percent) = $free =~ /(\d+)%/;
311 warn "almost out of space" if $percent > 90;
312 The other methods return a boolean value indicating successful
314 execution of the command on the remote machine.
315 Command line switches can be passed along with other arguments:
317 $volume->mkdir('-p','media/photos/vacation');
319 $volume->chown('-R','fred','media');
320 With the exception of df, each of these commands runs as the superuser,
322 so be careful how you call them.
323 You may run your own commands using the cmd() and ssh() methods. The
325 former returns the output of the command. The latter returns a success
326 code:
327 @log = $volume->cmd('tar cvf /tmp/archive.tar .');
329 $volume->ssh('gzip /tmp/archive.tar') or die "couldn't compress archive";
330 Before calling any of these methods, the volume must be mounted and its
332 server running. A fatal error will occur otherwise.
333
335 VM::EC2 VM::EC2::Staging::Manager VM::EC2::Staging::Server
336 VM::EC2::Instance VM::EC2::Volume VM::EC2::Snapshot
337
339 Lincoln Stein <lincoln.stein@gmail.com>.
340 Copyright (c) 2012 Ontario Institute for Cancer Research
342 This package and its accompanying libraries is free software; you can
344 redistribute it and/or modify it under the terms of the GPL (either
345 version 1, or at your option, any later version) or the Artistic
346 License 2.0. Refer to LICENSE for the full license text. In addition,
347 please see DISCLAIMER.txt for disclaimers of warranty.
348perl v5.38.0 2023-07-21 VM::EC2::Staging::Volume(3pm)