1VM::EC2::Staging::ManagUesre(r3)Contributed Perl DocumenVtMa:t:iEoCn2::Staging::Manager(3)
2
6 VM::EC2::Staging::Manager - Automate VMs and volumes for moving data in
7 and out of cloud.
8
10 use VM::EC2::Staging::Manager;
11 my $ec2 = VM::EC2->new(-region=>'us-east-1');
13 my $staging = $ec2->staging_manager(-on_exit => 'stop', # default, stop servers when process exists
14 -verbose => 1, # default, verbose progress messages
15 -scan => 1, # default, scan region for existing staging servers and volumes
16 -image_name => 'ubuntu-precise-12.04', # default server image
17 -user_name => 'ubuntu', # default server login name
18 );
19 # Assuming an EBS image named ami-12345 is located in the US, copy it into
21 # the South American region, returning the AMI ID in South America
22 my $new_image = $staging->copy_image('ami-12345','sa-east-1');
23 # provision a new server, using defaults. Name will be assigned automatically
25 my $server = $staging->provision_server(-availability_zone => 'us-east-1a');
26 # retrieve a new server named "my_server", if one exists. If not, it creates one
28 # using the specified options
29 my $server = $staging->get_server(-name => 'my_server',
30 -availability_zone => 'us-east-1a',
31 -instance_type => 't1.micro');
32 # open up an ssh session in an xterm
34 $server->shell;
35 # run a command over ssh on the server. See VM::EC2::Staging::Server
37 $server->ssh('whoami');
38 # run a command over ssh on the server, returning the result as an array of lines or a
40 # scalar string, similar to backticks (``)
41 my @password_lines = $server->scmd('cat /etc/passwd');
42 # run a command on the server and read from it using a filehandle
44 my $fh = $server->scmd_read('ls -R /usr/lib');
45 while (<$fh>) { # do something }
46 # run a command on the server and write to it using a filehandle
48 my $fh = $server->scmd_write('sudo -s "cat >>/etc/fstab"');
49 print $fh "/dev/sdf3 /mnt/demo ext3 0 2\n";
50 close $fh;
51 # Provision a new volume named "Pictures". Will automatically be mounted to a staging server in
53 # the specified zone. Server will be created if needed.
54 my $volume = $staging->provision_volume(-name => 'Pictures',
55 -fstype => 'ext4',
56 -availability_zone => 'us-east-1a',
57 -size => 2) or die $staging->error_str;
58 # gets an existing volume named "Pictures" if it exists. Otherwise provisions a new volume;
60 my $volume = $staging->get_volume(-name => 'Pictures',
61 -fstype => 'ext4',
62 -availability_zone => 'us-east-1a',
63 -size => 2) or die $staging->error_str;
64 # copy contents of local directory /opt/test to remote volume $volume using rsync
66 # See VM::EC2::Staging::Volume
67 $volume->put('/opt/test/');
68 # same thing, but first creating a subdirectory on the remote volume
70 $volume->put('/opt/test/' => './mirrors/');
71 # copy contents of remote volume $volume to local directory /tmp/test using rsync
73 $volume->get('/tmp/test');
74 # same thing, but from a subdirectory of the remote volume
76 $volume->get('./mirrors/' => '/tmp/test');
77 # server to server transfer (works both within and between availability regions)
79 my $south_america = VM::EC2->new(-region=>'sa-east-1')->staging_manager; # create a staging manager in Sao Paolo
80 my $volume2 = $south_america->provision_volume(-name => 'Videos',
81 -availability_zone => 'sa-east-1a',
82 -size => 2);
83 $staging->rsync("$volume/mirrors" => "$volume2/us-east");
84 $staging->stop_all_servers();
86 $staging->start_all_servers();
87 $staging->terminate_all_servers();
88 $staging->force_terminate_all_servers();
89
91 VM::EC2::Staging::Manager manages a set of EC2 volumes and servers in a
92 single AWS region. It was primarily designed to simplify the process of
93 provisioning and populating volumes, but it also provides a handy set
94 of ssh commands that allow you to run remote commands programmatically.
95 The manager also allows you to copy EBS-backed AMIs and their attached
97 volumes from one region to another, something that is otherwise
98 difficult to do.
99 The main classes are:
101 VM::EC2::Staging::Manager -- A set of volume and server resources in
103 a single AWS region.
104 VM::EC2::Staging::Server -- A staging server running somewhere in the
106 region. It is a VM::EC2::Instance
107 extended to provide remote command and
108 copy facilities.
109 VM::EC2::Staging::Volume -- A staging disk volume running somewhere in the
111 region. It is a VM::EC2::Volume
112 extended to provide remote copy
113 facilities.
114 Staging servers can provision volumes, format them, mount them, copy
116 data between local and remote (virtual) machines, and execute secure
117 shell commands. Staging volumes can mount themselves on servers, run a
118 variety of filesystem-oriented commands, and invoke commands on the
119 servers to copy data around locally and remotely.
120 See VM::EC2::Staging::Server and VM::EC2::Staging::Volume for the full
122 details.
123
125 The following methods allow you to create new VM::EC2::Staging::Manager
126 instances. Be aware that only one manager is allowed per EC2 region;
127 attempting to create additional managers in the same region will return
128 the same one each time.
129 $manager = $ec2->staging_manager(@args)
131 This is a simplified way to create a staging manager. First create the
132 EC2 object in the desired region, and then call its staging_manager()
133 method:
134 $manager = VM::EC2->new(-region=>'us-west-2')->staging_manager()
136 The staging_manager() method is only known to VM::EC2 objects if you
138 first "use" VM::EC2::Staging::Manager.
139 Required Arguments
141 None.
142 Optional Arguments
144 The optional arguments change the way that the manager creates new
145 servers and volumes.
146 -on_exit What to do with running servers when the manager goes
148 out of scope or the script exits. One of 'run',
149 'stop' (default), or 'terminate'. "run" keeps all
150 created instances running, so beware!
151 -architecture Architecture for newly-created server
153 instances (default "i386"). Can be overridden in calls to get_server()
154 and provision_server().
155 -instance_type Type of newly-created servers (default "m1.small"). Can be overridden
157 in calls to get_server() and provision_server().
158 -root_type Root type for newly-created servers (default depends
160 on the -on_exit behavior; "ebs" for exit behavior of
161 "stop" and "instance-store" for exit behavior of "run"
162 or "terminate".
163 -image_name Name or ami ID of the AMI to use for creating the
165 instances of new servers. Defaults to 'ubuntu-precise-12.04'.
166 If the image name begins with "ami-", then it is
167 treated as an AMI ID. Otherwise it is treated as
168 a name pattern and will be used to search the AMI
169 name field using the wildcard search "*$name*".
170 Names work better than AMI ids here, because the
171 latter change from one region to another. If multiple
172 matching image candidates are found, then an alpha
173 sort on the name is used to find the image with the
174 highest alpha sort value, which happens to work with
175 Ubuntu images to find the latest release.
176 -availability_zone Availability zone for newly-created
178 servers. Default is undef, in which case a random
179 zone is selected.
180 -username Username to use for ssh connections. Defaults to
182 "ubuntu". Note that this user must be able to use
183 sudo on the instance without providing a password,
184 or functionality of this module will be limited.
185 -verbose Integer level of verbosity. Level 1 prints warning
187 messages. Level 2 (the default) adds informational
188 messages as well. Level 3 adds verbose debugging
189 messages. Level 0 suppresses all messages.
190 -quiet (deprecated) If true, turns off all verbose messages.
192 -scan Boolean, default true. If true, scans region for
194 volumes and servers created by earlier manager
195 instances.
196 -reuse_key Boolean, default true. If true, creates a single
198 ssh keypair for each region and reuses it. Note that
199 the private key is kept on the local computer in the
200 directory ~/.vm-ec2-staging, and so additional
201 keypairs may be created if you use this module on
202 multiple local machines. If this option is false,
203 then a new keypair will be created for every server
204 you partition.
205 -reuse_volumes Boolean, default true. If this flag is true, then
207 calls to provision_volume() will return existing
208 volumes if they share the same name as the requested
209 volume. If no suitable existing volume exists, then
210 the most recent snapshot of this volume is used to
211 create it in the specified availability zone. Only
212 if no volume or snapshot exist will a new volume be
213 created from scratch.
214 -dotdir Path to the directory that contains keyfiles and other
216 stable configuration information for this module.
217 Defaults to ~/.vm_ec2_staging. You may wish to change
218 this to, say, a private dropbox directory or an NFS-mount
219 in order to share keyfiles among machines. Be aware of
220 the security implications of sharing private key files.
221 -server_class By default, staging server objects created by the manager
223 are of class type VM::EC2::Staging::Server. If you create
224 a custom server subclass, you need to let the manager know
225 about it by passing the class name to this argument.
226 -volume_class By default, staging volume objects created by the manager
228 are of class type VM::EC2::Staging::Volume. If you create
229 a custom volume subclass, you need to let the manager know
230 about it by passing the class name to this argument.
231 $manager = VM::EC2::Staging::Manager(-ec2 => $ec2,@args)
233 This is a more traditional constructur for the staging manager.
234 Required Arguments
236 -ec2 A VM::EC2 object.
237 Optional Arguments
239 All of the arguments listed in the description of
240 VM::EC2->staging_manager().
241
243 This library provides convenience methods for copying whole AMIs as
244 well as individual snapshots from one zone to another. It does this by
245 gathering information about the AMI/snapshot in the source zone,
246 creating staging servers in the source and target zones, and then
247 copying the volume data from the source to the target. If an
248 AMI/snapshot does not use a recognized filesystem (e.g. it is part of
249 an LVM or RAID disk set), then block level copying of the entire device
250 is used. Otherwise, rsync() is used to minimize data transfer fees.
251 Note that interzone copying of instance-backed AMIs is not supported.
253 Only EBS-backed images can be copied in this way.
254 See also the command-line script migrate-ebs-image.pl that comes with
256 this package.
257 $new_image_id =
259 $manager->copy_image($source_image,$destination_zone,@register_options)
260 This method copies the AMI indicated by $source_image from the zone
261 that $manager belongs to, into the indicated $destination_zone, and
262 returns the AMI ID of the new image in the destination zone.
263 $source_image may be an AMI ID, or a VM::EC2::Image object.
265 $destination_zone may be a simple region name, such as "us-west-2", or
267 a VM::EC2::Region object (as returned by VM::EC2->describe_regions), or
268 a VM::EC2::Staging::Manager object that is associated with the desired
269 region. The latter form gives you control over the nature of the
270 staging instances created in the destination zone. For example, if you
271 wish to use 'm1.large' high-I/O instances in both the source and
272 destination reasons, you would proceed like this:
273 my $source = VM::EC2->new(-region=>'us-east-1'
275 )->staging_manager(-instance_type=>'m1.large',
276 -on_exit =>'terminate');
277 my $destination = VM::EC2->new(-region=>'us-west-2'
278 )->staging_manager(-instance_type=>'m1.large',
279 -on_exit =>'terminate');
280 my $new_image = $source->copy_image('ami-123456' => $destination);
281 If present, the named argument list @register_options will be passed to
283 register_image() and used to override options in the destination image.
284 This can be used to set ephemeral device mappings, which cannot
285 currently be detected and transferred automatically by copy_image():
286 $new_image =$source->copy_image('ami-123456' => 'us-west-2',
288 -description => 'My AMI western style',
289 -block_devices => '/dev/sde=ephemeral0');
290 $dest_kernel = $manager->match_kernel($src_kernel,$dest_zone)
292 Find a kernel in $dest_zone that matches the $src_kernel in the current
293 zone. $dest_zone can be a VM::EC2::Staging manager object, a region
294 name, or a VM::EC2::Region object.
295 $new_snapshot_id =
297 $manager->copy_snapshot($source_snapshot,$destination_zone)
298 This method copies the EBS snapshot indicated by $source_snapshot from
299 the zone that $manager belongs to, into the indicated
300 $destination_zone, and returns the ID of the new snapshot in the
301 destination zone.
302 $source_snapshot may be an string ID, or a VM::EC2::Snapshot object.
304 $destination_zone may be a simple region name, such as "us-west-2", or
306 a VM::EC2::Region object (as returned by VM::EC2->describe_regions), or
307 a VM::EC2::Staging::Manager object that is associated with the desired
308 region.
309 Note that this call uses the Amazon CopySnapshot API call that was
311 introduced in 2012-12-01 and no longer involves the creation of staging
312 servers in the source and destination regions.
313
315 These methods allow you to create and interrogate staging servers. They
316 each return one or more VM::EC2::Staging::Server objects. See
317 VM::EC2::Staging::Server for more information about what you can do
318 with these servers once they are running.
319 $server = $manager->provision_server(%options)
321 Create a new VM::EC2::Staging::Server object according to the passed
322 options, which override the default options provided by the Manager
323 object.
324 -name Name for this server, which can be used to retrieve
326 it later with a call to get_server().
327 -architecture Architecture for the newly-created server
329 instances (e.g. "i386"). If not specified, then defaults
330 to the default_architecture() value. If explicitly
331 specified as undef, then the architecture of the matching
332 image will be used.
333 -instance_type Type of the newly-created server (e.g. "m1.small").
335 -root_type Root type for the server ("ebs" or "instance-store").
337 -image_name Name or ami ID of the AMI to use for creating the
339 instance for the server. If the image name begins with
340 "ami-", then it is treated as an AMI ID. Otherwise it
341 is treated as a name pattern and will be used to
342 search the AMI name field using the wildcard search
343 "*$name*". Names work better than AMI ids here,
344 because the latter change from one region to
345 another. If multiple matching image candidates are
346 found, then an alpha sort on the name is used to find
347 the image with the highest alpha sort value, which
348 happens to work with Ubuntu images to find the latest
349 release.
350 -availability_zone Availability zone for the server, or undef to
352 choose an availability zone randomly.
353 -username Username to use for ssh connections. Defaults to
355 "ubuntu". Note that this user must be able to use
356 sudo on the instance without providing a password,
357 or functionality of this server will be limited.
358 In addition, you may use any of the options recognized by
360 VM::EC2->run_instances() (e.g. -block_devices).
361 $server = $manager->get_server(-name=>$name,%other_options)
363 $server = $manager->get_server($name)
364 Return an existing VM::EC2::Staging::Server object having the indicated
365 symbolic name, or create a new server if one with this name does not
366 already exist. The server's instance characteristics will be configured
367 according to the options passed to the manager at create time (e.g.
368 -availability_zone, -instance_type). These options can be overridden by
369 %other_args. See provision_volume() for details.
370 $server =
372 $manager->get_server_in_zone(-zone=>$availability_zone,%other_options)
373 $server = $manager->get_server_in_zone($availability_zone)
374 Return an existing VM::EC2::Staging::Server running in the indicated
375 symbolic name, or create a new server if one with this name does not
376 already exist. The server's instance characteristics will be configured
377 according to the options passed to the manager at create time (e.g.
378 -availability_zone, -instance_type). These options can be overridden by
379 %other_args. See provision_server() for details.
380 $server = $manager->find_server_by_instance($instance_id)
382 Given an EC2 instanceId, return the corresponding
383 VM::EC2::Staging::Server, if any.
384 @servers $manager->servers
386 Return all registered VM::EC2::Staging::Servers in the zone managed by
387 the manager.
388 $manager->start_all_servers
390 Start all VM::EC2::Staging::Servers that are currently in the "stop"
391 state.
392 $manager->stop_all_servers
394 Stop all VM::EC2::Staging::Servers that are currently in the "running"
395 state.
396 $manager->terminate_all_servers
398 Terminate all VM::EC2::Staging::Servers and unregister them.
399 $manager->force_terminate_all_servers
401 Force termination of all VM::EC2::Staging::Servers, even if the
402 internal registration system indicates that some may be in use by other
403 Manager instances.
404 $manager->wait_for_servers(@servers)
406 Wait until all the servers on the list @servers are up and able to
407 accept ssh commands. You may wish to wrap this in an eval{} and timeout
408 in order to avoid waiting indefinitely.
409
411 These methods allow you to create and interrogate staging volumes. They
412 each return one or more VM::EC2::Staging::Volume objects. See
413 VM::EC2::Staging::Volume for more information about what you can do
414 with these staging volume objects.
415 $volume = $manager->provision_volume(%options)
417 Create and register a new VM::EC2::Staging::Volume and mount it on a
418 staging server in the appropriate availability zone. A new staging
419 server will be created for this purpose if one does not already exist.
420 If you provide a symbolic name for the volume and the manager has
422 previously snapshotted a volume by the same name, then the snapshot
423 will be used to create the volume (this behavior can be suppressed by
424 passing -reuse=>0). This allows for the following pattern for
425 efficiently updating a snapshotted volume:
426 my $vol = $manager->provision_volume(-name=>'MyPictures',
428 -size=>10);
429 $vol->put('/usr/local/my_pictures/'); # will do an rsync from local directory
430 $vol->create_snapshot; # write out to a snapshot
431 $vol->delete;
432 You may also explicitly specify a volumeId or snapshotId. The former
434 allows you to place an existing volume under management of
435 VM::EC2::Staging::Manager and returns a corresponding staging volume
436 object. The latter creates the staging volume from the indicated
437 snapshot, irregardless of whether the snapshot was created by the
438 staging manager at an earlier time.
439 Newly-created staging volumes are automatically formatted as ext4
441 filesystems and mounted on the staging server under /mnt/Staging/$name,
442 where $name is the staging volume's symbolic name. The filesystem type
443 and the mountpoint can be modified with the -fstype and -mount
444 arguments, respectively. In addition, you may specify an -fstype of
445 "raw", in which case the volume will be attached to a staging server
446 (creating the server first if necessary) but not formatted or mounted.
447 This is useful when creating multi-volume RAID or LVM setups.
448 Options:
450 -name Name of the staging volume. A fatal error issues if a staging
452 volume by this name already exists (use get_volume() to
453 avoid this). If no name is provided, then a random
454 unique one is chosen for you.
455 -availability_zone
457 Availability zone in which to create this
458 volume. If none is specified, then a zone is chosen that
459 reuses an existing staging server, if any.
460 -size Size of the desired volume, in GB.
462 -fstype Filesystem type for the volume, ext4 by default. Supported
464 types are ext2, ext3, ext4, xfs, reiserfs, jfs, hfs,
465 ntfs, vfat, msdos, and raw.
466 -mount Mount point for this volume on the staging server (e.g. /opt/bin).
468 Use with care, as there are no checks to prevent you from mounting
469 two staging volumes on top of each other or mounting over essential
470 operating system paths.
471 -label Volume label. Only applies to filesystems that support labels
473 (all except hfs, vfat, msdos and raw).
474 -volume_id Create the staging volume from an existing EBS volume with
476 the specified ID. Most other options are ignored in this
477 case.
478 -snapshot_id
480 Create the staging volume from an existing EBS
481 snapshot. If a size is specified that is larger than the
482 snapshot, then the volume and its filesystem will be
483 automatically extended (this only works for ext volumes
484 at the moment). Shrinking of volumes is not currently
485 supported.
486 -reuse If true, then the most recent snapshot created from a staging
488 volume of the same name is used to create the
489 volume. This is the default. Pass 0 to disable this
490 behavior.
491 The -reuse argument is intended to support the following use case in
493 which you wish to rsync a directory on a host system somewhere to an
494 EBS snapshot, without maintaining a live server and volume on EC2:
495 my $volume = $manager->provision_volume(-name=>'backup_1',
497 -reuse => 1,
498 -fstype => 'ext3',
499 -size => 10);
500 $volume->put('fred@gw.harvard.edu:my_music');
501 $volume->create_snapshot('Music Backup '.localtime);
502 $volume->delete;
503 The next time this script is run, the "backup_1" volume will be
505 recreated from the most recent snapshot, minimizing copying. A new
506 snapshot is created, and the staging volume is deleted.
507 $volume = $manager->get_volume(-name=>$name,%other_options)
509 $volume = $manager->get_volume($name)
510 Return an existing VM::EC2::Staging::Volume object with the indicated
511 symbolic name, or else create a new volume if one with this name does
512 not already exist. The volume's characteristics will be configured
513 according to the options in %other_args. See provision_volume() for
514 details. If called with no arguments, this method returns Volume object
515 with default characteristics and a randomly-assigned name.
516 $result = $manager->rsync($src1,$src2,$src3...,$dest)
518 This method provides remote synchronization (rsync) file-level copying
519 between one or more source locations and a destination location via an
520 ssh tunnel. Copying among arbitrary combinations of local and remote
521 filesystems is supported, with the caveat that the remote filesystems
522 must be contained on volumes and servers managed by this module (see
523 below for a workaround).
524 You may provide two or more directory paths. The last path will be
526 treated as the copy destination, and the source paths will be treated
527 as copy sources. All copying is performed using the -avz options, which
528 activates recursive directory copying in which ownership, modification
529 times and permissions are preserved, and compresses the data to reduce
530 network usage. Verbosity is set so that the names of copied files are
531 printed to STDERR. If you do not wish this, then use call the manager's
532 quiet() method with a true value.
533 Source paths can be formatted in one of several ways:
535 /absolute/path
537 Copy the contents of the directory /absolute/path located on the
538 local machine to the destination. This will create a
539 subdirectory named "path" on the destination disk. Add a slash
540 to the end of the path (i.e. "/absolute/path/") in order to
541 avoid creating this subdirectory on the destination disk.
542 ./relative/path
544 Relative paths work the way you expect, and depend on the current
545 working directory. The terminating slash rule applies.
546 $staging_volume
548 Pass a VM::EC2::Staging::Volume to copy the contents of the
549 volume to the destination disk starting at the root of the
550 volume. Note that you do *not* need to have any knowledge of the
551 mount point for this volume in order to copy its contents.
552 $staging_volume:/absolute/path
554 $staging_volume:absolute/path
555 $staging_volume/absolute/path
556 All these syntaxes accomplish the same thing, which is to
557 copy a subdirectory of a staging volume to the destination disk.
558 The root of the volume is its top level, regardless of where it
559 is mounted on the staging server. Because of string
560 interpolation magic, you can enclose staging volume object names
561 in quotes in order to construct the path, as in
562 "$picture_volume:/family/vacations/". As in local paths, a
563 terminating slash indicates that the contents of the last
564 directory in the path are to be copied without creating the
565 enclosing directory on the desetination. Note that you do *not*
566 need to have any knowledge of the mount point for this volume in
567 order to copy its contents.
568 $staging_server:/absolute/path
570 Pass a staging server object and absolute path to copy the contents
571 of this path to the destination disk. Because of string interpolation
572 you can include server objects in quotes: "$my_server:/opt"
573 $staging_server:relative/path
575 This form will copy data from paths relative to the remote user's home
576 directory on the staging server. Typically not very useful, but supported.
577 The same syntax is supported for destination paths, except that it
579 makes no difference whether a path has a trailing slash or not.
580 As with the rsync command, if you proceed a path with a single colon
582 (:/my/path), it is a short hand to use the previous server/volume/host
583 in the source list.
584 When specifying multiple source directories, all source directories
586 must reside on the same local or remote machine. This is legal:
587 $manager->rsync("$picture_volume:/family/vacations",
589 "$picture_volume:/family/picnics"
590 => "$backup_volume:/recent_backups");
591 This is not:
593 $manager->rsync("$picture_volume:/family/vacations",
595 "$audio_volume:/beethoven"
596 => "$backup_volume:/recent_backups");
597 When specifying multiple sources, you may give the volume or server
599 once for the first source and then start additional source paths with a
600 ":" to indicate the same volume or server is to be used:
601 $manager->rsync("$picture_volume:/family/vacations",
603 ":/family/picnics"
604 => "$backup_volume:/recent_backups");
605 When copying to/from the local machine, the rsync process will run as
607 the user that the script was launched by. However, on remote servers
608 managed by the staging manager, the rsync process will run as
609 superuser.
610 The rsync() method will also accept regular remote DNS names and IP
612 addresses, optionally preceded by a username:
613 $manager->rsync("$picture_volume:/family/vacations" => 'fred@gw.harvard.edu:/tmp')
615 When called in this way, the method does what it can to avoid prompting
617 for a password or passphrase on the non-managed host (gw.harvard.edu in
618 the above example). This includes turning off strict host checking and
619 forwarding the user agent information from the local machine.
620 $result = $manager->rsync(\@options,$src1,$src2,$src3...,$dest)
622 This is a variant of the rsync command in which extra options can be
623 passed to rsync by providing an array reference as the first argument.
624 For example:
625 $manager->rsync(['--exclude' => '*~'],
627 '/usr/local/backups',
628 "$my_server:/usr/local");
629 $manager->dd($source_vol=>$dest_vol)
631 This method performs block-level copying of the contents of $source_vol
632 to $dest_vol by using dd over an SSH tunnel, where both source and
633 destination volumes are VM::EC2::Staging::Volume objects. The volumes
634 must be attached to a server but not mounted. Everything in the volume,
635 including its partition table, is copied, allowing you to make an exact
636 image of a disk.
637 The volumes do not actually need to reside on this server, but can be
639 attached to any staging server in the zone.
640 $volume = $manager->find_volume_by_volid($volume_id)
642 Given an EC2 volumeId, return the corresponding
643 VM::EC2::Staging::Volume, if any.
644 $volume = $manager->find_volume_by_name($name)
646 Given a staging name (assigned at volume creation time), return the
647 corresponding VM::EC2::Staging::Volume, if any.
648 @volumes = $manager->volumes
650 Return all VM::EC2::Staging::Volumes managed in this zone.
651
653 This section documents accessor methods that allow you to examine or
654 change configuration options that were set at create time. Called with
655 an argument, the accessor changes the option and returns the option's
656 previous value. Called without an argument, the accessor returns the
657 option's current value.
658 $on_exit = $manager->on_exit([$new_behavior])
660 Get or set the "on_exit" option, which specifies what to do with
661 existing staging servers when the staging manager is destroyed. Valid
662 values are "terminate", "stop" and "run".
663 $reuse_key = $manager->reuse_key([$boolean])
665 Get or set the "reuse_key" option, which if true uses the same
666 internally-generated ssh keypair for all running instances. If false,
667 then a new keypair will be created for each staging server. The keypair
668 will be destroyed automatically when the staging server terminates (but
669 only if the staging manager initiates the termination itself).
670 $username = $manager->username([$new_username])
672 Get or set the username used to log into staging servers.
673 $architecture = $manager->architecture([$new_architecture])
675 Get or set the architecture (i386, x86_64) to use for launching new
676 staging servers.
677 $root_type = $manager->root_type([$new_type])
679 Get or set the instance root type for new staging servers ("instance-
680 store", "ebs").
681 $instance_type = $manager->instance_type([$new_type])
683 Get or set the instance type to use for new staging servers (e.g.
684 "t1.micro"). I recommend that you use "m1.small" (the default) or
685 larger instance types because of the extremely slow I/O of the micro
686 instance. In addition, micro instances running Ubuntu have a known bug
687 that prevents them from unmounting and remounting EBS volumes
688 repeatedly on the same block device. This can lead to hangs when the
689 staging manager tries to create volumes.
690 $reuse_volumes = $manager->reuse_volumes([$new_boolean])
692 This gets or sets the "reuse_volumes" option, which if true causes the
693 provision_volumes() call to create staging volumes from existing EBS
694 volumes and snapshots that share the same staging manager symbolic
695 name. See the discussion under VM::EC2->staging_manager(), and
696 VM::EC2::Staging::Manager->provision_volume().
697 $name = $manager->image_name([$new_name])
699 This gets or sets the "image_name" option, which is the AMI ID or AMI
700 name to use when creating new staging servers. Names beginning with
701 "ami-" are treated as AMI IDs, and everything else is treated as a
702 pattern match on the AMI name.
703 $zone = $manager->availability_zone([$new_zone])
705 Get or set the default availability zone to use when creating new
706 servers and volumes. An undef value allows the staging manager to
707 choose the zone in a way that minimizes resources.
708 $class_name = $manager->volume_class([$new_class])
710 Get or set the name of the perl package that implements staging
711 volumes, VM::EC2::Staging::Volume by default. Staging volumes created
712 by the manager will have this class type.
713 $class_name = $manager->server_class([$new_class])
715 Get or set the name of the perl package that implements staging
716 servers, VM::EC2::Staging::Server by default. Staging servers created
717 by the manager will have this class type.
718 $boolean = $manager->scan([$boolean])
720 Get or set the "scan" flag, which if true will cause the zone to be
721 scanned quickly for existing managed servers and volumes when the
722 manager is first created.
723 $path = $manager->dot_directory([$new_directory])
725 Get or set the dot directory which holds private key files.
726
728 This section documents internal methods that are not normally called by
729 end-user scripts but may be useful in subclasses. In addition, there
730 are a number of undocumented internal methods that begin with the "_"
731 character. Explore the source code to learn about these.
732 $ok = $manager->environment_ok
734 This performs a check on the environment in which the module is
735 running. For this module to work properly, the ssh, rsync and dd
736 programs must be found in the PATH. If all three programs are found,
737 then this method returns true.
738 This method can be called as an instance method or class method.
740 $name = $manager->default_verbosity
742 Returns the default verbosity level (2: warning+informational
743 messages). This is overridden using -verbose at create time.
744 $name = $manager->default_exit_behavior
746 Return the default exit behavior ("stop") when the manager terminates.
747 Intended to be overridden in subclasses.
748 $name = $manager->default_image_name
750 Return the default image name ('ubuntu-precise-12.04') for use in
751 creating new instances. Intended to be overridden in subclasses.
752 $name = $manager->default_user_name
754 Return the default user name ('ubuntu') for use in creating new
755 instances. Intended to be overridden in subclasses.
756 $name = $manager->default_architecture
758 Return the default instance architecture ('i386') for use in creating
759 new instances. Intended to be overridden in subclasses.
760 $name = $manager->default_root_type
762 Return the default instance root type ('instance-store') for use in
763 creating new instances. Intended to be overridden in subclasses. Note
764 that this value is ignored if the exit behavior is "stop", in which
765 case an ebs-backed instance will be used. Also, the m1.micro instance
766 type does not come in an instance-store form, so ebs will be used in
767 this case as well.
768 $name = $manager->default_instance_type
770 Return the default instance type ('m1.small') for use in creating new
771 instances. Intended to be overridden in subclasses. We default to
772 m1.small rather than a micro instance because the I/O in m1.small is
773 far faster than in t1.micro.
774 $name = $manager->default_reuse_keys
776 Return the default value of the -reuse_keys argument ('true'). This
777 value allows the manager to create an ssh keypair once, and use the
778 same one for all servers it creates over time. If false, then a new
779 keypair is created for each server and then discarded when the server
780 terminates.
781 $name = $manager->default_reuse_volumes
783 Return the default value of the -reuse_volumes argument ('true'). This
784 value instructs the manager to use the symbolic name of the volume to
785 return an existing volume whenever a request is made to provision a new
786 one of the same name.
787 $path = $manager->default_dot_directory_path
789 Return the default value of the -dotdir argument
790 ("$ENV{HOME}/.vm-ec2-staging"). This value instructs the manager to use
791 the symbolic name of the volume to return an existing volume whenever a
792 request is made to provision a new one of the same name.
793 $class_name = $manager->default_volume_class
795 Return the class name for staging volumes created by the manager,
796 VM::EC2::Staging::Volume by default. If you wish a subclass of
797 VM::EC2::Staging::Manager to create a different type of volume,
798 override this method.
799 $class_name = $manager->default_server_class
801 Return the class name for staging servers created by the manager,
802 VM::EC2::Staging::Server by default. If you wish a subclass of
803 VM::EC2::Staging::Manager to create a different type of volume,
804 override this method.
805 $server = $manager->register_server($server)
807 Register a VM::EC2::Staging::Server object. Usually called internally.
808 $manager->unregister_server($server)
810 Forget about the existence of VM::EC2::Staging::Server. Usually called
811 internally.
812 $manager->register_volume($volume)
814 Register a VM::EC2::Staging::Volume object. Usually called internally.
815 $manager->unregister_volume($volume)
817 Forget about a VM::EC2::Staging::Volume object. Usually called
818 internally.
819 $pid = $manager->pid([$new_pid])
821 Get or set the process ID of the script that is running the manager.
822 This is used internally to detect the case in which the script has
823 forked, in which case we do not want to invoke the manager class's
824 destructor in the child process (because it may stop or terminate
825 servers still in use by the parent process).
826 $path = $manager->dotdir([$new_dotdir])
828 Low-level version of dot_directory(), differing only in the fact that
829 dot_directory will automatically create the path, including
830 subdirectories.
831 $manager->scan_region
833 Synchronize internal list of managed servers and volumes with the EC2
834 region. Called automatically during new() and needed only if servers &
835 volumes are changed from outside the module while it is running.
836 $group = $manager->security_group
838 Returns or creates a security group with the permissions needed used to
839 manage staging servers. Usually called internally.
840 $keypair = $manager->keypair
842 Returns or creates the ssh keypair used internally by the manager to to
843 access staging servers. Usually called internally.
844 $name = $manager->new_volume_name
846 Returns a new random name for volumes provisioned without a -name
847 argument. Currently names are in of the format "volume-12345678", where
848 the numeric part are 8 random hex digits. Although no attempt is made
849 to prevent naming collisions, the large number of possible names makes
850 this unlikely.
851 $name = $manager->new_server_name
853 Returns a new random name for server provisioned without a -name
854 argument. Currently names are in of the format "server-12345678", where
855 the numeric part are 8 random hex digits. Although no attempt is made
856 to prevent naming collisions, the large number of possible names makes
857 this unlikely.
858 $description = $manager->volume_description($volume)
860 This method is called to assign a description to newly-created volumes.
861 The current format is "Staging volume for Foo created by
862 VM::EC2::Staging::Manager", where Foo is the volume's symbolic name.
863 $manager->debug("Debugging message\n")
865 $manager->info("Informational message\n")
866 $manager->warn("Warning message\n")
867 Prints an informational message to standard error if current
868 verbosity() level allows.
869 $verbosity = $manager->verbosity([$new_value])
871 The verbosity() method get/sets a flag that sets the level of
872 informational messages.
873
875 VM::EC2 VM::EC2::Staging::Server VM::EC2::Staging::Volume
876 migrate-ebs-image.pl
877
879 Lincoln Stein <lincoln.stein@gmail.com>.
880 Copyright (c) 2012 Ontario Institute for Cancer Research
882 This package and its accompanying libraries is free software; you can
884 redistribute it and/or modify it under the terms of the GPL (either
885 version 1, or at your option, any later version) or the Artistic
886 License 2.0. Refer to LICENSE for the full license text. In addition,
887 please see DISCLAIMER.txt for disclaimers of warranty.
888perl v5.32.0 2020-07-28 VM::EC2::Staging::Manager(3)