1VM::EC2::Staging::ManagUesre(r3)Contributed Perl DocumenVtMa:t:iEoCn2::Staging::Manager(3)
2
3
4

NAME

6       VM::EC2::Staging::Manager - Automate VMs and volumes for moving data in
7       and out of cloud.
8

SYNOPSIS

10        use VM::EC2::Staging::Manager;
11
12        my $ec2     = VM::EC2->new(-region=>'us-east-1');
13        my $staging = $ec2->staging_manager(-on_exit     => 'stop', # default, stop servers when process exists
14                                            -verbose     => 1,      # default, verbose progress messages
15                                            -scan        => 1,      # default, scan region for existing staging servers and volumes
16                                            -image_name  => 'ubuntu-precise-12.04',  # default server image
17                                            -user_name   => 'ubuntu',                # default server login name
18                                            );
19
20        # Assuming an EBS image named ami-12345 is located in the US, copy it into
21        # the South American region, returning the AMI ID in South America
22        my $new_image = $staging->copy_image('ami-12345','sa-east-1');
23
24        # provision a new server, using defaults. Name will be assigned automatically
25        my $server = $staging->provision_server(-availability_zone => 'us-east-1a');
26
27        # retrieve a new server named "my_server", if one exists. If not, it creates one
28        # using the specified options
29        my $server = $staging->get_server(-name              => 'my_server',
30                                          -availability_zone => 'us-east-1a',
31                                          -instance_type     => 't1.micro');
32
33        # open up an ssh session in an xterm
34        $server->shell;
35
36        # run a command over ssh on the server. See VM::EC2::Staging::Server
37        $server->ssh('whoami');
38
39        # run a command over ssh on the server, returning the result as an array of lines or a
40        # scalar string, similar to backticks (``)
41        my @password_lines = $server->scmd('cat /etc/passwd');
42
43        # run a command on the server and read from it using a filehandle
44        my $fh  = $server->scmd_read('ls -R /usr/lib');
45        while (<$fh>) { # do something }
46
47        # run a command on the server and write to it using a filehandle
48        my $fh  = $server->scmd_write('sudo -s "cat >>/etc/fstab"');
49        print $fh "/dev/sdf3 /mnt/demo ext3 0 2\n";
50        close $fh;
51
52        # Provision a new volume named "Pictures". Will automatically be mounted to a staging server in
53        # the specified zone. Server will be created if needed.
54        my $volume = $staging->provision_volume(-name              => 'Pictures',
55                                                -fstype            => 'ext4',
56                                                -availability_zone => 'us-east-1a',
57                                                -size              => 2) or die $staging->error_str;
58
59        # gets an existing volume named "Pictures" if it exists. Otherwise provisions a new volume;
60        my $volume = $staging->get_volume(-name              => 'Pictures',
61                                          -fstype            => 'ext4',
62                                          -availability_zone => 'us-east-1a',
63                                          -size              => 2) or die $staging->error_str;
64
65        # copy contents of local directory /opt/test to remote volume $volume using rsync
66        # See VM::EC2::Staging::Volume
67        $volume->put('/opt/test/');
68
69        # same thing, but first creating a subdirectory on the remote volume
70        $volume->put('/opt/test/' => './mirrors/');
71
72        # copy contents of remote volume $volume to local directory /tmp/test using rsync
73        $volume->get('/tmp/test');
74
75        # same thing, but from a subdirectory of the remote volume
76        $volume->get('./mirrors/' => '/tmp/test');
77
78        # server to server transfer (works both within and between availability regions)
79        my $south_america = VM::EC2->new(-region=>'sa-east-1')->staging_manager;    # create a staging manager in Sao Paolo
80        my $volume2 = $south_america->provision_volume(-name              => 'Videos',
81                                                       -availability_zone => 'sa-east-1a',
82                                                       -size              => 2);
83        $staging->rsync("$volume/mirrors" => "$volume2/us-east");
84
85        $staging->stop_all_servers();
86        $staging->start_all_servers();
87        $staging->terminate_all_servers();
88        $staging->force_terminate_all_servers();
89

DESCRIPTION

91       VM::EC2::Staging::Manager manages a set of EC2 volumes and servers in a
92       single AWS region. It was primarily designed to simplify the process of
93       provisioning and populating volumes, but it also provides a handy set
94       of ssh commands that allow you to run remote commands programmatically.
95
96       The manager also allows you to copy EBS-backed AMIs and their attached
97       volumes from one region to another, something that is otherwise
98       difficult to do.
99
100       The main classes are:
101
102        VM::EC2::Staging::Manager -- A set of volume and server resources in
103                                     a single AWS region.
104
105        VM::EC2::Staging::Server -- A staging server running somewhere in the
106                                    region. It is a VM::EC2::Instance
107                                    extended to provide remote command and
108                                    copy facilities.
109
110        VM::EC2::Staging::Volume -- A staging disk volume running somewhere in the
111                                    region. It is a VM::EC2::Volume
112                                    extended to provide remote copy
113                                    facilities.
114
115       Staging servers can provision volumes, format them, mount them, copy
116       data between local and remote (virtual) machines, and execute secure
117       shell commands. Staging volumes can mount themselves on servers, run a
118       variety of filesystem-oriented commands, and invoke commands on the
119       servers to copy data around locally and remotely.
120
121       See VM::EC2::Staging::Server and VM::EC2::Staging::Volume for the full
122       details.
123

Constructors

125       The following methods allow you to create new VM::EC2::Staging::Manager
126       instances. Be aware that only one manager is allowed per EC2 region;
127       attempting to create additional managers in the same region will return
128       the same one each time.
129
130   $manager = $ec2->staging_manager(@args)
131       This is a simplified way to create a staging manager. First create the
132       EC2 object in the desired region, and then call its staging_manager()
133       method:
134
135        $manager = VM::EC2->new(-region=>'us-west-2')->staging_manager()
136
137       The staging_manager() method is only known to VM::EC2 objects if you
138       first "use" VM::EC2::Staging::Manager.
139
140       Required Arguments
141           None.
142
143       Optional Arguments
144           The optional arguments change the way that the manager creates new
145           servers and volumes.
146
147            -on_exit       What to do with running servers when the manager goes
148                           out of scope or the script exits. One of 'run',
149                           'stop' (default), or 'terminate'. "run" keeps all
150                           created instances running, so beware!
151
152            -architecture  Architecture for newly-created server
153                           instances (default "i386"). Can be overridden in calls to get_server()
154                           and provision_server().
155
156            -instance_type Type of newly-created servers (default "m1.small"). Can be overridden
157                           in calls to get_server() and provision_server().
158
159            -root_type     Root type for newly-created servers (default depends
160                           on the -on_exit behavior; "ebs" for exit behavior of
161                           "stop" and "instance-store" for exit behavior of "run"
162                           or "terminate".
163
164            -image_name    Name or ami ID of the AMI to use for creating the
165                           instances of new servers. Defaults to 'ubuntu-precise-12.04'.
166                           If the image name begins with "ami-", then it is
167                           treated as an AMI ID. Otherwise it is treated as
168                           a name pattern and will be used to search the AMI
169                           name field using the wildcard search "*$name*".
170                           Names work better than AMI ids here, because the
171                           latter change from one region to another. If multiple
172                           matching image candidates are found, then an alpha
173                           sort on the name is used to find the image with the
174                           highest alpha sort value, which happens to work with
175                           Ubuntu images to find the latest release.
176
177            -availability_zone Availability zone for newly-created
178                           servers. Default is undef, in which case a random
179                           zone is selected.
180
181            -username      Username to use for ssh connections. Defaults to
182                           "ubuntu". Note that this user must be able to use
183                           sudo on the instance without providing a password,
184                           or functionality of this module will be limited.
185
186            -verbose       Integer level of verbosity. Level 1 prints warning
187                           messages. Level 2 (the default) adds informational
188                           messages as well. Level 3 adds verbose debugging
189                           messages. Level 0 suppresses all messages.
190
191            -quiet         (deprecated) If true, turns off all verbose messages.
192
193            -scan          Boolean, default true. If true, scans region for
194                           volumes and servers created by earlier manager
195                           instances.
196
197            -reuse_key     Boolean, default true. If true, creates a single
198                           ssh keypair for each region and reuses it. Note that
199                           the private key is kept on the local computer in the
200                           directory ~/.vm-ec2-staging, and so additional
201                           keypairs may be created if you use this module on
202                           multiple local machines. If this option is false,
203                           then a new keypair will be created for every server
204                           you partition.
205
206            -reuse_volumes Boolean, default true. If this flag is true, then
207                           calls to provision_volume() will return existing
208                           volumes if they share the same name as the requested
209                           volume. If no suitable existing volume exists, then
210                           the most recent snapshot of this volume is used to
211                           create it in the specified availability zone. Only
212                           if no volume or snapshot exist will a new volume be
213                           created from scratch.
214
215            -dotdir        Path to the directory that contains keyfiles and other
216                           stable configuration information for this module.
217                           Defaults to ~/.vm_ec2_staging. You may wish to change
218                           this to, say, a private dropbox directory or an NFS-mount
219                           in order to share keyfiles among machines. Be aware of
220                           the security implications of sharing private key files.
221
222            -server_class  By default, staging server objects created by the manager
223                           are of class type VM::EC2::Staging::Server. If you create
224                           a custom server subclass, you need to let the manager know
225                           about it by passing the class name to this argument.
226
227            -volume_class  By default, staging volume objects created by the manager
228                           are of class type VM::EC2::Staging::Volume. If you create
229                           a custom volume subclass, you need to let the manager know
230                           about it by passing the class name to this argument.
231
232   $manager = VM::EC2::Staging::Manager(-ec2 => $ec2,@args)
233       This is a more traditional constructur for the staging manager.
234
235       Required Arguments
236             -ec2     A VM::EC2 object.
237
238       Optional Arguments
239           All of the arguments listed in the description of
240           VM::EC2->staging_manager().
241

Interzone Copying of AMIs and Snapshots

243       This library provides convenience methods for copying whole AMIs as
244       well as individual snapshots from one zone to another. It does this by
245       gathering information about the AMI/snapshot in the source zone,
246       creating staging servers in the source and target zones, and then
247       copying the volume data from the source to the target. If an
248       AMI/snapshot does not use a recognized filesystem (e.g. it is part of
249       an LVM or RAID disk set), then block level copying of the entire device
250       is used. Otherwise, rsync() is used to minimize data transfer fees.
251
252       Note that interzone copying of instance-backed AMIs is not supported.
253       Only EBS-backed images can be copied in this way.
254
255       See also the command-line script migrate-ebs-image.pl that comes with
256       this package.
257
258   $new_image_id =
259       $manager->copy_image($source_image,$destination_zone,@register_options)
260       This method copies the AMI indicated by $source_image from the zone
261       that $manager belongs to, into the indicated $destination_zone, and
262       returns the AMI ID of the new image in the destination zone.
263
264       $source_image may be an AMI ID, or a VM::EC2::Image object.
265
266       $destination_zone may be a simple region name, such as "us-west-2", or
267       a VM::EC2::Region object (as returned by VM::EC2->describe_regions), or
268       a VM::EC2::Staging::Manager object that is associated with the desired
269       region. The latter form gives you control over the nature of the
270       staging instances created in the destination zone. For example, if you
271       wish to use 'm1.large' high-I/O instances in both the source and
272       destination reasons, you would proceed like this:
273
274        my $source      = VM::EC2->new(-region=>'us-east-1'
275                                      )->staging_manager(-instance_type=>'m1.large',
276                                                         -on_exit      =>'terminate');
277        my $destination = VM::EC2->new(-region=>'us-west-2'
278                                      )->staging_manager(-instance_type=>'m1.large',
279                                                         -on_exit      =>'terminate');
280        my $new_image   = $source->copy_image('ami-123456' => $destination);
281
282       If present, the named argument list @register_options will be passed to
283       register_image() and used to override options in the destination image.
284       This can be used to set ephemeral device mappings, which cannot
285       currently be detected and transferred automatically by copy_image():
286
287        $new_image =$source->copy_image('ami-123456'   => 'us-west-2',
288                                        -description   => 'My AMI western style',
289                                        -block_devices => '/dev/sde=ephemeral0');
290
291   $dest_kernel = $manager->match_kernel($src_kernel,$dest_zone)
292       Find a kernel in $dest_zone that matches the $src_kernel in the current
293       zone. $dest_zone can be a VM::EC2::Staging manager object, a region
294       name, or a VM::EC2::Region object.
295
296   $new_snapshot_id =
297       $manager->copy_snapshot($source_snapshot,$destination_zone)
298       This method copies the EBS snapshot indicated by $source_snapshot from
299       the zone that $manager belongs to, into the indicated
300       $destination_zone, and returns the ID of the new snapshot in the
301       destination zone.
302
303       $source_snapshot may be an string ID, or a VM::EC2::Snapshot object.
304
305       $destination_zone may be a simple region name, such as "us-west-2", or
306       a VM::EC2::Region object (as returned by VM::EC2->describe_regions), or
307       a VM::EC2::Staging::Manager object that is associated with the desired
308       region.
309
310       Note that this call uses the Amazon CopySnapshot API call that was
311       introduced in 2012-12-01 and no longer involves the creation of staging
312       servers in the source and destination regions.
313

Instance Methods for Managing Staging Servers

315       These methods allow you to create and interrogate staging servers. They
316       each return one or more VM::EC2::Staging::Server objects. See
317       VM::EC2::Staging::Server for more information about what you can do
318       with these servers once they are running.
319
320   $server = $manager->provision_server(%options)
321       Create a new VM::EC2::Staging::Server object according to the passed
322       options, which override the default options provided by the Manager
323       object.
324
325        -name          Name for this server, which can be used to retrieve
326                       it later with a call to get_server().
327
328        -architecture  Architecture for the newly-created server
329                       instances (e.g. "i386"). If not specified, then defaults
330                       to the default_architecture() value. If explicitly
331                       specified as undef, then the architecture of the matching
332                       image will be used.
333
334        -instance_type Type of the newly-created server (e.g. "m1.small").
335
336        -root_type     Root type for the server ("ebs" or "instance-store").
337
338        -image_name    Name or ami ID of the AMI to use for creating the
339                       instance for the server. If the image name begins with
340                       "ami-", then it is treated as an AMI ID. Otherwise it
341                       is treated as a name pattern and will be used to
342                       search the AMI name field using the wildcard search
343                       "*$name*". Names work better than AMI ids here,
344                       because the latter change from one region to
345                       another. If multiple matching image candidates are
346                       found, then an alpha sort on the name is used to find
347                       the image with the highest alpha sort value, which
348                       happens to work with Ubuntu images to find the latest
349                       release.
350
351        -availability_zone Availability zone for the server, or undef to
352                       choose an availability zone randomly.
353
354        -username      Username to use for ssh connections. Defaults to
355                       "ubuntu". Note that this user must be able to use
356                       sudo on the instance without providing a password,
357                       or functionality of this server will be limited.
358
359       In addition, you may use any of the options recognized by
360       VM::EC2->run_instances() (e.g. -block_devices).
361
362   $server = $manager->get_server(-name=>$name,%other_options)
363   $server = $manager->get_server($name)
364       Return an existing VM::EC2::Staging::Server object having the indicated
365       symbolic name, or create a new server if one with this name does not
366       already exist. The server's instance characteristics will be configured
367       according to the options passed to the manager at create time (e.g.
368       -availability_zone, -instance_type). These options can be overridden by
369       %other_args. See provision_volume() for details.
370
371   $server =
372       $manager->get_server_in_zone(-zone=>$availability_zone,%other_options)
373   $server = $manager->get_server_in_zone($availability_zone)
374       Return an existing VM::EC2::Staging::Server running in the indicated
375       symbolic name, or create a new server if one with this name does not
376       already exist. The server's instance characteristics will be configured
377       according to the options passed to the manager at create time (e.g.
378       -availability_zone, -instance_type). These options can be overridden by
379       %other_args. See provision_server() for details.
380
381   $server = $manager->find_server_by_instance($instance_id)
382       Given an EC2 instanceId, return the corresponding
383       VM::EC2::Staging::Server, if any.
384
385   @servers $manager->servers
386       Return all registered VM::EC2::Staging::Servers in the zone managed by
387       the manager.
388
389   $manager->start_all_servers
390       Start all VM::EC2::Staging::Servers that are currently in the "stop"
391       state.
392
393   $manager->stop_all_servers
394       Stop all VM::EC2::Staging::Servers that are currently in the "running"
395       state.
396
397   $manager->terminate_all_servers
398       Terminate all VM::EC2::Staging::Servers and unregister them.
399
400   $manager->force_terminate_all_servers
401       Force termination of all VM::EC2::Staging::Servers, even if the
402       internal registration system indicates that some may be in use by other
403       Manager instances.
404
405   $manager->wait_for_servers(@servers)
406       Wait until all the servers on the list @servers are up and able to
407       accept ssh commands. You may wish to wrap this in an eval{} and timeout
408       in order to avoid waiting indefinitely.
409

Instance Methods for Managing Staging Volumes

411       These methods allow you to create and interrogate staging volumes. They
412       each return one or more VM::EC2::Staging::Volume objects. See
413       VM::EC2::Staging::Volume for more information about what you can do
414       with these staging volume objects.
415
416   $volume = $manager->provision_volume(%options)
417       Create and register a new VM::EC2::Staging::Volume and mount it on a
418       staging server in the appropriate availability zone. A new staging
419       server will be created for this purpose if one does not already exist.
420
421       If you provide a symbolic name for the volume and the manager has
422       previously snapshotted a volume by the same name, then the snapshot
423       will be used to create the volume (this behavior can be suppressed by
424       passing -reuse=>0). This allows for the following pattern for
425       efficiently updating a snapshotted volume:
426
427        my $vol = $manager->provision_volume(-name=>'MyPictures',
428                                             -size=>10);
429        $vol->put('/usr/local/my_pictures/');   # will do an rsync from local directory
430        $vol->create_snapshot;  # write out to a snapshot
431        $vol->delete;
432
433       You may also explicitly specify a volumeId or snapshotId. The former
434       allows you to place an existing volume under management of
435       VM::EC2::Staging::Manager and returns a corresponding staging volume
436       object. The latter creates the staging volume from the indicated
437       snapshot, irregardless of whether the snapshot was created by the
438       staging manager at an earlier time.
439
440       Newly-created staging volumes are automatically formatted as ext4
441       filesystems and mounted on the staging server under /mnt/Staging/$name,
442       where $name is the staging volume's symbolic name. The filesystem type
443       and the mountpoint can be modified with the -fstype and -mount
444       arguments, respectively. In addition, you may specify an -fstype of
445       "raw", in which case the volume will be attached to a staging server
446       (creating the server first if necessary) but not formatted or mounted.
447       This is useful when creating multi-volume RAID or LVM setups.
448
449       Options:
450
451        -name       Name of the staging volume. A fatal error issues if a staging
452                    volume by this name already exists (use get_volume() to
453                    avoid this).  If no name is provided, then a random
454                    unique one is chosen for you.
455
456        -availability_zone
457                    Availability zone in which to create this
458                    volume. If none is specified, then a zone is chosen that
459                    reuses an existing staging server, if any.
460
461        -size       Size of the desired volume, in GB.
462
463        -fstype     Filesystem type for the volume, ext4 by default. Supported
464                    types are ext2, ext3, ext4, xfs, reiserfs, jfs, hfs,
465                    ntfs, vfat, msdos, and raw.
466
467        -mount      Mount point for this volume on the staging server (e.g. /opt/bin).
468                    Use with care, as there are no checks to prevent you from mounting
469                    two staging volumes on top of each other or mounting over essential
470                    operating system paths.
471
472        -label      Volume label. Only applies to filesystems that support labels
473                    (all except hfs, vfat, msdos and raw).
474
475        -volume_id  Create the staging volume from an existing EBS volume with
476                    the specified ID. Most other options are ignored in this
477                    case.
478
479        -snapshot_id
480                    Create the staging volume from an existing EBS
481                    snapshot. If a size is specified that is larger than the
482                    snapshot, then the volume and its filesystem will be
483                    automatically extended (this only works for ext volumes
484                    at the moment). Shrinking of volumes is not currently
485                    supported.
486
487        -reuse      If true, then the most recent snapshot created from a staging
488                    volume of the same name is used to create the
489                    volume. This is the default. Pass 0 to disable this
490                    behavior.
491
492       The -reuse argument is intended to support the following use case in
493       which you wish to rsync a directory on a host system somewhere to an
494       EBS snapshot, without maintaining a live server and volume on EC2:
495
496        my $volume = $manager->provision_volume(-name=>'backup_1',
497                                                -reuse  => 1,
498                                                -fstype => 'ext3',
499                                                -size   => 10);
500        $volume->put('fred@gw.harvard.edu:my_music');
501        $volume->create_snapshot('Music Backup '.localtime);
502        $volume->delete;
503
504       The next time this script is run, the "backup_1" volume will be
505       recreated from the most recent snapshot, minimizing copying. A new
506       snapshot is created, and the staging volume is deleted.
507
508   $volume = $manager->get_volume(-name=>$name,%other_options)
509   $volume = $manager->get_volume($name)
510       Return an existing VM::EC2::Staging::Volume object with the indicated
511       symbolic name, or else create a new volume if one with this name does
512       not already exist. The volume's characteristics will be configured
513       according to the options in %other_args. See provision_volume() for
514       details. If called with no arguments, this method returns Volume object
515       with default characteristics and a randomly-assigned name.
516
517   $result = $manager->rsync($src1,$src2,$src3...,$dest)
518       This method provides remote synchronization (rsync) file-level copying
519       between one or more source locations and a destination location via an
520       ssh tunnel. Copying among arbitrary combinations of local and remote
521       filesystems is supported, with the caveat that the remote filesystems
522       must be contained on volumes and servers managed by this module (see
523       below for a workaround).
524
525       You may provide two or more directory paths. The last path will be
526       treated as the copy destination, and the source paths will be treated
527       as copy sources. All copying is performed using the -avz options, which
528       activates recursive directory copying in which ownership, modification
529       times and permissions are preserved, and compresses the data to reduce
530       network usage. Verbosity is set so that the names of copied files are
531       printed to STDERR. If you do not wish this, then use call the manager's
532       quiet() method with a true value.
533
534       Source paths can be formatted in one of several ways:
535
536        /absolute/path
537             Copy the contents of the directory /absolute/path located on the
538             local machine to the destination. This will create a
539             subdirectory named "path" on the destination disk. Add a slash
540             to the end of the path (i.e. "/absolute/path/") in order to
541             avoid creating this subdirectory on the destination disk.
542
543        ./relative/path
544             Relative paths work the way you expect, and depend on the current
545             working directory. The terminating slash rule applies.
546
547        $staging_volume
548             Pass a VM::EC2::Staging::Volume to copy the contents of the
549             volume to the destination disk starting at the root of the
550             volume. Note that you do *not* need to have any knowledge of the
551             mount point for this volume in order to copy its contents.
552
553        $staging_volume:/absolute/path
554        $staging_volume:absolute/path
555        $staging_volume/absolute/path
556             All these syntaxes accomplish the same thing, which is to
557             copy a subdirectory of a staging volume to the destination disk.
558             The root of the volume is its top level, regardless of where it
559             is mounted on the staging server.  Because of string
560             interpolation magic, you can enclose staging volume object names
561             in quotes in order to construct the path, as in
562             "$picture_volume:/family/vacations/". As in local paths, a
563             terminating slash indicates that the contents of the last
564             directory in the path are to be copied without creating the
565             enclosing directory on the desetination. Note that you do *not*
566             need to have any knowledge of the mount point for this volume in
567             order to copy its contents.
568
569        $staging_server:/absolute/path
570            Pass a staging server object and absolute path to copy the contents
571            of this path to the destination disk. Because of string interpolation
572            you can include server objects in quotes: "$my_server:/opt"
573
574        $staging_server:relative/path
575            This form will copy data from paths relative to the remote user's home
576            directory on the staging server. Typically not very useful, but supported.
577
578       The same syntax is supported for destination paths, except that it
579       makes no difference whether a path has a trailing slash or not.
580
581       As with the rsync command, if you proceed a path with a single colon
582       (:/my/path), it is a short hand to use the previous server/volume/host
583       in the source list.
584
585       When specifying multiple source directories, all source directories
586       must reside on the same local or remote machine. This is legal:
587
588        $manager->rsync("$picture_volume:/family/vacations",
589                        "$picture_volume:/family/picnics"
590                        => "$backup_volume:/recent_backups");
591
592       This is not:
593
594        $manager->rsync("$picture_volume:/family/vacations",
595                        "$audio_volume:/beethoven"
596                        => "$backup_volume:/recent_backups");
597
598       When specifying multiple sources, you may give the volume or server
599       once for the first source and then start additional source paths with a
600       ":" to indicate the same volume or server is to be used:
601
602        $manager->rsync("$picture_volume:/family/vacations",
603                        ":/family/picnics"
604                        => "$backup_volume:/recent_backups");
605
606       When copying to/from the local machine, the rsync process will run as
607       the user that the script was launched by. However, on remote servers
608       managed by the staging manager, the rsync process will run as
609       superuser.
610
611       The rsync() method will also accept regular remote DNS names and IP
612       addresses, optionally preceded by a username:
613
614        $manager->rsync("$picture_volume:/family/vacations" => 'fred@gw.harvard.edu:/tmp')
615
616       When called in this way, the method does what it can to avoid prompting
617       for a password or passphrase on the non-managed host (gw.harvard.edu in
618       the above example). This includes turning off strict host checking and
619       forwarding the user agent information from the local machine.
620
621   $result = $manager->rsync(\@options,$src1,$src2,$src3...,$dest)
622       This is a variant of the rsync command in which extra options can be
623       passed to rsync by providing an array reference as the first argument.
624       For example:
625
626           $manager->rsync(['--exclude' => '*~'],
627                           '/usr/local/backups',
628                           "$my_server:/usr/local");
629
630   $manager->dd($source_vol=>$dest_vol)
631       This method performs block-level copying of the contents of $source_vol
632       to $dest_vol by using dd over an SSH tunnel, where both source and
633       destination volumes are VM::EC2::Staging::Volume objects. The volumes
634       must be attached to a server but not mounted. Everything in the volume,
635       including its partition table, is copied, allowing you to make an exact
636       image of a disk.
637
638       The volumes do not actually need to reside on this server, but can be
639       attached to any staging server in the zone.
640
641   $volume = $manager->find_volume_by_volid($volume_id)
642       Given an EC2 volumeId, return the corresponding
643       VM::EC2::Staging::Volume, if any.
644
645   $volume = $manager->find_volume_by_name($name)
646       Given a staging name (assigned at volume creation time), return the
647       corresponding VM::EC2::Staging::Volume, if any.
648
649   @volumes = $manager->volumes
650       Return all VM::EC2::Staging::Volumes managed in this zone.
651

Instance Methods for Accessing Configuration Options

653       This section documents accessor methods that allow you to examine or
654       change configuration options that were set at create time. Called with
655       an argument, the accessor changes the option and returns the option's
656       previous value. Called without an argument, the accessor returns the
657       option's current value.
658
659   $on_exit = $manager->on_exit([$new_behavior])
660       Get or set the "on_exit" option, which specifies what to do with
661       existing staging servers when the staging manager is destroyed. Valid
662       values are "terminate", "stop" and "run".
663
664   $reuse_key = $manager->reuse_key([$boolean])
665       Get or set the "reuse_key" option, which if true uses the same
666       internally-generated ssh keypair for all running instances. If false,
667       then a new keypair will be created for each staging server. The keypair
668       will be destroyed automatically when the staging server terminates (but
669       only if the staging manager initiates the termination itself).
670
671   $username = $manager->username([$new_username])
672       Get or set the username used to log into staging servers.
673
674   $architecture = $manager->architecture([$new_architecture])
675       Get or set the architecture (i386, x86_64) to use for launching new
676       staging servers.
677
678   $root_type = $manager->root_type([$new_type])
679       Get or set the instance root type for new staging servers ("instance-
680       store", "ebs").
681
682   $instance_type = $manager->instance_type([$new_type])
683       Get or set the instance type to use for new staging servers (e.g.
684       "t1.micro"). I recommend that you use "m1.small" (the default) or
685       larger instance types because of the extremely slow I/O of the micro
686       instance. In addition, micro instances running Ubuntu have a known bug
687       that prevents them from unmounting and remounting EBS volumes
688       repeatedly on the same block device. This can lead to hangs when the
689       staging manager tries to create volumes.
690
691   $reuse_volumes = $manager->reuse_volumes([$new_boolean])
692       This gets or sets the "reuse_volumes" option, which if true causes the
693       provision_volumes() call to create staging volumes from existing EBS
694       volumes and snapshots that share the same staging manager symbolic
695       name. See the discussion under VM::EC2->staging_manager(), and
696       VM::EC2::Staging::Manager->provision_volume().
697
698   $name = $manager->image_name([$new_name])
699       This gets or sets the "image_name" option, which is the AMI ID or AMI
700       name to use when creating new staging servers. Names beginning with
701       "ami-" are treated as AMI IDs, and everything else is treated as a
702       pattern match on the AMI name.
703
704   $zone = $manager->availability_zone([$new_zone])
705       Get or set the default availability zone to use when creating new
706       servers and volumes. An undef value allows the staging manager to
707       choose the zone in a way that minimizes resources.
708
709   $class_name = $manager->volume_class([$new_class])
710       Get or set the name of the perl package that implements staging
711       volumes, VM::EC2::Staging::Volume by default. Staging volumes created
712       by the manager will have this class type.
713
714   $class_name = $manager->server_class([$new_class])
715       Get or set the name of the perl package that implements staging
716       servers, VM::EC2::Staging::Server by default. Staging servers created
717       by the manager will have this class type.
718
719   $boolean = $manager->scan([$boolean])
720       Get or set the "scan" flag, which if true will cause the zone to be
721       scanned quickly for existing managed servers and volumes when the
722       manager is first created.
723
724   $path = $manager->dot_directory([$new_directory])
725       Get or set the dot directory which holds private key files.
726

Internal Methods

728       This section documents internal methods that are not normally called by
729       end-user scripts but may be useful in subclasses. In addition, there
730       are a number of undocumented internal methods that begin with the "_"
731       character. Explore the source code to learn about these.
732
733   $ok   = $manager->environment_ok
734       This performs a check on the environment in which the module is
735       running. For this module to work properly, the ssh, rsync and dd
736       programs must be found in the PATH. If all three programs are found,
737       then this method returns true.
738
739       This method can be called as an instance method or class method.
740
741   $name = $manager->default_verbosity
742       Returns the default verbosity level (2: warning+informational
743       messages). This is overridden using -verbose at create time.
744
745   $name = $manager->default_exit_behavior
746       Return the default exit behavior ("stop") when the manager terminates.
747       Intended to be overridden in subclasses.
748
749   $name = $manager->default_image_name
750       Return the default image name ('ubuntu-precise-12.04') for use in
751       creating new instances. Intended to be overridden in subclasses.
752
753   $name = $manager->default_user_name
754       Return the default user name ('ubuntu') for use in creating new
755       instances. Intended to be overridden in subclasses.
756
757   $name = $manager->default_architecture
758       Return the default instance architecture ('i386') for use in creating
759       new instances. Intended to be overridden in subclasses.
760
761   $name = $manager->default_root_type
762       Return the default instance root type ('instance-store') for use in
763       creating new instances. Intended to be overridden in subclasses. Note
764       that this value is ignored if the exit behavior is "stop", in which
765       case an ebs-backed instance will be used. Also, the m1.micro instance
766       type does not come in an instance-store form, so ebs will be used in
767       this case as well.
768
769   $name = $manager->default_instance_type
770       Return the default instance type ('m1.small') for use in creating new
771       instances. Intended to be overridden in subclasses. We default to
772       m1.small rather than a micro instance because the I/O in m1.small is
773       far faster than in t1.micro.
774
775   $name = $manager->default_reuse_keys
776       Return the default value of the -reuse_keys argument ('true'). This
777       value allows the manager to create an ssh keypair once, and use the
778       same one for all servers it creates over time. If false, then a new
779       keypair is created for each server and then discarded when the server
780       terminates.
781
782   $name = $manager->default_reuse_volumes
783       Return the default value of the -reuse_volumes argument ('true'). This
784       value instructs the manager to use the symbolic name of the volume to
785       return an existing volume whenever a request is made to provision a new
786       one of the same name.
787
788   $path = $manager->default_dot_directory_path
789       Return the default value of the -dotdir argument
790       ("$ENV{HOME}/.vm-ec2-staging"). This value instructs the manager to use
791       the symbolic name of the volume to return an existing volume whenever a
792       request is made to provision a new one of the same name.
793
794   $class_name = $manager->default_volume_class
795       Return the class name for staging volumes created by the manager,
796       VM::EC2::Staging::Volume by default. If you wish a subclass of
797       VM::EC2::Staging::Manager to create a different type of volume,
798       override this method.
799
800   $class_name = $manager->default_server_class
801       Return the class name for staging servers created by the manager,
802       VM::EC2::Staging::Server by default. If you wish a subclass of
803       VM::EC2::Staging::Manager to create a different type of volume,
804       override this method.
805
806   $server = $manager->register_server($server)
807       Register a VM::EC2::Staging::Server object. Usually called internally.
808
809   $manager->unregister_server($server)
810       Forget about the existence of VM::EC2::Staging::Server. Usually called
811       internally.
812
813   $manager->register_volume($volume)
814       Register a VM::EC2::Staging::Volume object. Usually called internally.
815
816   $manager->unregister_volume($volume)
817       Forget about a VM::EC2::Staging::Volume object. Usually called
818       internally.
819
820   $pid = $manager->pid([$new_pid])
821       Get or set the process ID of the script that is running the manager.
822       This is used internally to detect the case in which the script has
823       forked, in which case we do not want to invoke the manager class's
824       destructor in the child process (because it may stop or terminate
825       servers still in use by the parent process).
826
827   $path = $manager->dotdir([$new_dotdir])
828       Low-level version of dot_directory(), differing only in the fact that
829       dot_directory will automatically create the path, including
830       subdirectories.
831
832   $manager->scan_region
833       Synchronize internal list of managed servers and volumes with the EC2
834       region. Called automatically during new() and needed only if servers &
835       volumes are changed from outside the module while it is running.
836
837   $group = $manager->security_group
838       Returns or creates a security group with the permissions needed used to
839       manage staging servers. Usually called internally.
840
841   $keypair = $manager->keypair
842       Returns or creates the ssh keypair used internally by the manager to to
843       access staging servers. Usually called internally.
844
845   $name = $manager->new_volume_name
846       Returns a new random name for volumes provisioned without a -name
847       argument. Currently names are in of the format "volume-12345678", where
848       the numeric part are 8 random hex digits. Although no attempt is made
849       to prevent naming collisions, the large number of possible names makes
850       this unlikely.
851
852   $name = $manager->new_server_name
853       Returns a new random name for server provisioned without a -name
854       argument. Currently names are in of the format "server-12345678", where
855       the numeric part are 8 random hex digits.  Although no attempt is made
856       to prevent naming collisions, the large number of possible names makes
857       this unlikely.
858
859   $description = $manager->volume_description($volume)
860       This method is called to assign a description to newly-created volumes.
861       The current format is "Staging volume for Foo created by
862       VM::EC2::Staging::Manager", where Foo is the volume's symbolic name.
863
864   $manager->debug("Debugging message\n")
865   $manager->info("Informational message\n")
866   $manager->warn("Warning message\n")
867       Prints an informational message to standard error if current
868       verbosity() level allows.
869
870   $verbosity = $manager->verbosity([$new_value])
871       The verbosity() method get/sets a flag that sets the level of
872       informational messages.
873

SEE ALSO

875       VM::EC2 VM::EC2::Staging::Server VM::EC2::Staging::Volume
876       migrate-ebs-image.pl
877

AUTHOR

879       Lincoln Stein <lincoln.stein@gmail.com>.
880
881       Copyright (c) 2012 Ontario Institute for Cancer Research
882
883       This package and its accompanying libraries is free software; you can
884       redistribute it and/or modify it under the terms of the GPL (either
885       version 1, or at your option, any later version) or the Artistic
886       License 2.0.  Refer to LICENSE for the full license text. In addition,
887       please see DISCLAIMER.txt for disclaimers of warranty.
888
889
890
891perl v5.36.0                      2022-07-22      VM::EC2::Staging::Manager(3)
Impressum