Importing VMDK to AWS using EC2 CLI

Yesterday I showed how we can import OVA file to AWS as an AMI image, today I'm going to demonstrate how we can import actual VMDK file that is exported from any virtualize software as an instance in EC2.

Before we started, there are prerequisites to be done first.

1. Setting up EC2 CLI to work with your terminal. Test it with simple command:

> ec2-describe-regions

2. Create S3 bucket, name it anything you want. Mine is "instance-manifest". This bucket will save our VMDK's manifest and small VMDK parts then convert them to EC2 instance.

3. Now use this command to start importing VMDK:

> ec2-import-instance -o <Access Key> -w <Secret Key> -t <Instance Type> -a x86_64 -f <File Type> -p <Platform> -b <Bucket name> -s <EBS disk size> -z <Availability Zone> "\path\to\file.vmdk"

Change to red highlight to appropriated value to suit your environment.

Example Command:

> ec2-import-instance -o Your_Access_Key -w Your_Secret_Key -t t2.micro -a x86_64 -f VMDK -p Linux -b instance-manifest -s 10 -z ap-southeast-1a "D:\Downloads\ubuntu-disk1-streamed.vmdk"

4. Let's check to status of importing with the following command:

> ec2-describe-conversion-tasks --region <region_name>

or

> ec2-describe-conversion-tasks <task_id>

5. When the importing is done. Go to AWS console, EC2 then you'll see new instance from conversion.

Setup SSH Public Key Credentials for imported instances

In fact, there are many prerequisites to be done before exporting VMDK file and import it into AWS. One of them is to configure a non-root user to use public key-based SSH to access your instance after it is imported. 

I haven't done it yet, so here's the thing I did after I imported my VM.

1. Login to instance using password method.
2. Create public key for my root account using the following script:

if [ ! -d /root/.ssh ] ; then
        mkdir -p /root/.ssh
        chmod 700 /root/.ssh
fi
# Fetch public key using HTTP
curl http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key > /tmp/my-key
if [ $? -eq 0 ] ; then
        cat /tmp/my-key >> /root/.ssh/authorized_keys
        chmod 700 /root/.ssh/authorized_keys
        rm /tmp/my-key
fi

***This script can be applied to any user account.

3. Create private key from given public key pair you downloaded when you create an instance using putty gen.
4. Login with that private key and edit "/etc/ssh/sshd_config" file with these following conditions:

RSAAuthentication yes
PublicKeyAuthentication yes
AuthorizedKeysFile %h/.ssh/authorized_keys
PasswordAuthentication no
UsePAM no
PermitRootLogin without-password

5. Or you can create your own key for each account with the following command:

l2oxAz@localhost:~/$ ssh-keygen -t rsal2oxAz@localhost:~/$ ssh-copy-id l2oxAz@127.0.0.1

Importing OVA to AWS as AMI using AWS CLI (Part 2 of 2)

Okay, Now we are going to import our OVA files using AWS CLI tool. I've already created S3 bucket in my AWS account and uploaded OVA files also, put it in bucket named "exported-vmdk"

Let's open our terminal/cmd console or whatever console you're using with AWS CLI and type the following command to import OVA file and convert it into AMI image.

Here's my example of the command:

>aws ec2 import-image --cli-input-json "{  \"Description\": \"Ubuntu OVA\", \"DiskContainers\": [ { \"Description\": \"First CLI task\", \"UserBucket\": { \"S3Bucket\": \"exported-vmdk\", \"S3Key\" : \"ubuntu.ova\" } } ]}"

****S3Bucket is the bucket name on S3, S3Key is the name of the OVA file you're going to import
****Don't forget to change the command to suit with your environment.


Now If it successes, you can track the importing status by using the given image ID with this command:

>aws ec2 describe-import-image-tasks –import-task-ids <image-ID>

Once everything is done, login to AWS console, go to EC2 and look for AMI section. There will be an AMI image wait for you to create a new instance.

Importing OVA to AWS as AMI using AWS CLI (Part 1 of 2)

It's been awhile since my last post again. Now I'm busy with AWS and today I'm going to show you how we can upload our OVA file (exported from any virtualize vendor) to our S3 bucket and convert it to AMI which is an image format using by AWS.

Before we started, there are some prerequisites that needed to be done.

1. Download and install AWS CLI tool from AWS website, click here

(Don't forget to set your AWSCLI_HOME)

2. Now we need "VM Import Service Role" this will uses a role in your AWS account to perform certain operations. The role must be created with the name "vmimport" with the following policy and trust entities. Create a file name "trust-policy.json" with following policies:

{
   "Version":"2012-10-17",
   "Statement":[
      {
         "Sid":"",
         "Effect":"Allow",
         "Principal":{
            "Service":"vmie.amazonaws.com"
         },
         "Action":"sts:AssumeRole",
         "Condition":{
            "StringEquals":{
               "sts:ExternalId":"vmimport"
            }
         }
      }
   ]
} 

Then use this command to create service role:

> aws iam create-role --role-name vmimport --assume-role-policy-document file://trust-policy.json

3. Then create a file named "role-policy.json" to use as policy for this service role: 

{
   "Version":"2012-10-17",
   "Statement":[
      {
         "Effect":"Allow",
         "Action":[
            "s3:ListBucket",
            "s3:GetBucketLocation"
         ],
         "Resource":[
            "arn:aws:s3:::<disk-image-file-bucket>"
         ]
      },
      {
         "Effect":"Allow",
         "Action":[
            "s3:GetObject"
         ],
         "Resource":[
            "arn:aws:s3:::<disk-image-file-bucket>/*"
         ]
      },
      {
         "Effect":"Allow",
         "Action":[
            "ec2:ModifySnapshotAttribute",
            "ec2:CopySnapshot",
            "ec2:RegisterImage",
            "ec2:Describe*"
         ],
         "Resource":"*"
      }
   ]
}

Then create policy and replace <disk-image-file-bucket> with the appropriate Amazon S3 bucket where the disk files are stored. Run this command to attach the policy to the role created above:

> aws iam put-role-policy --role-name vmimport --policy-name vmimport --policy-document file://role-policy.json

4. If you're logged on as an IAM user, you'll need the following permissions in your IAM policy to import or export a VM:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListAllMyBuckets"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:CreateBucket",
        "s3:DeleteBucket",
        "s3:DeleteObject",
        "s3:GetBucketLocation",
        "s3:GetObject",
        "s3:ListBucket",
        "s3:PutObject"
      ],
      "Resource": ["arn:aws:s3:::mys3bucket","arn:aws:s3:::mys3bucket/*"]
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:CancelConversionTask",
        "ec2:CancelExportTask",
        "ec2:CreateImage",
        "ec2:CreateInstanceExportTask",
        "ec2:CreateTags",
        "ec2:DeleteTags",
        "ec2:DescribeConversionTasks",
        "ec2:DescribeExportTasks",
        "ec2:DescribeInstanceAttribute",
        "ec2:DescribeInstanceStatus",
        "ec2:DescribeInstances",
        "ec2:DescribeTags",
        "ec2:ImportInstance",
        "ec2:ImportVolume",
        "ec2:StartInstances",
        "ec2:StopInstances",
        "ec2:TerminateInstances",
        "ec2:ImportImage",
        "ec2:ImportSnapshot",
        "ec2:DescribeImportImageTasks",
        "ec2:DescribeImportSnapshotTasks",
        "ec2:CancelImportTask"
      ],
      "Resource": "*"
    }
  ]
}

Now we're ready to import our OVA files using AWS CLI tool. Let's continue on part 2...