Experience - Inspire - Contribute

CloudWatch an AWS service provides rich set of tools to monitor health and resource utilization of various services on AWS. These metrics can be used to trigger an alarm, send notifications, even trigger actions based on these alarms.

 

One main metric that is missing from default CloudWatch metrics is Memory utilization of EC2 instances. Since AWS does not have access to EC2 instance at the Operating System level, only CPU, Network utilization, IO and other metrics that can be monitored through Hypervisor layer are available by default in AWS Console.

 

AWS provides a set of scripts that utilize CloudWatch API to send Memory Utilization info and make use of CloudWatch features to monitor and take action if alarms are raised.

The process of sending these custom metrics are different for Linux and Windows Instances. Even the process of installing pre-requisites on different Linux distributions is slightly different.

 

In this post we will see the process of installing and configuring CloudWatch Custom Metrics for Redhat derived Linux Distros, mainly RHEL, Amazon Linux and CentOS along with Windows to monitor Memory Utilization of the EC2 instance

 

Updated (27-March-2016) – Added Commands for Ubuntu Linux

 

List of all commands run on Linux are also available at the end of the post for easy

reference.

 

Step –1

Create an IAM user with access key. We do not need password or console access to this user.

 

 

Copy and keep the Access key handy as we would need this every time we configure custom metrics to be sent to CloudWatch

 

 

Create and Attach an Inline Policy to the user with below actions.

cloudwatch:PutMetricData
cloudwatch:GetMetricStatistics
cloudwatch:ListMetrics
ec2:DescribeTags

 

Policy would like below. You may copy paste this for your use.

 

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["cloudwatch:PutMetricData", "cloudwatch:GetMetricStatistics", "cloudwatch:ListMetrics", "ec2:DescribeTags"],
      "Resource": "*"
    }
  ]
}

 

Go to IAM Users, click on the user just created and click on “click here” hyperlink in Inline Policies

 

 

In next page select custom Policy and click select

 

 

Give a name for Policy and paste policy in Policy Document and click on Apply Policy

 

 

Now the user would have policy attached to send custom metrics using API to CloudWatch

 

 

You can verify the policy using Show Policy Actions in IAM console for the user.

 

Step –2

Installing pre-requisites, all the following commands are to be run with root privilege, you can use sudo as well, except inside CPAN shell.

 

CEntOS 6.x -

 

Command -1

yum -y install perl-DateTime perl-Sys-Syslog openssl openssl-devel cpan unzip

 

Command -2

perl -MCPAN -e shell

 

If this command prompts for input, accept defaults

 

 

The above command when complete would show CPAN prompt

 

 

Run below command at CPAN prompt

 

Command -3

install Bundle::LWP6 LWP

 

This would prompt for input multiple times, accept all default values or answer yes. This would take some time to complete, so, have patience.

 

 

One the command finishes successfully, the CPAN prompt number would change, type quit and hit enter to exit to shell prompt

 

 

Run below command at shell prompt and accept all defaults or answer yes

 

Command -4

perl -MCPAN -e 'install LWP::Protocol::https'

 

Once this finishes successfully, you will have shell prompt waiting for next steps.

 

Step –3

Downloading Scripts

 

Run below commands to download, extract, delete the downloaded zip file and move scripts to /opt folder, and open awscreds.conf file to configure credentials.

 

wget http://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.1.zip

unzip CloudWatchMonitoringScripts-1.2.1.zip

rm -f CloudWatchMonitoringScripts-1.2.1.zip

mv aws-scripts-mon /opt/

cd /opt/aws-scripts-mon/

cp awscreds.template awscreds.conf

vi awscreds.conf

 

 

I have used vi for text editing, but you can use text editor of your choice.

 

Once awscreds.conf file is open in the text editor, key in Accesskey ID and Secret Key that we got when we created IAM user in Step-1

 

 

Save and exit text editor.

 

Now run below command to check if everything is OK.

 

/opt/aws-scripts-mon/mon-put-instance-data.pl --mem-util --verify –verbose

 

 

If you see the output as below, everything is properly configured and we can proceed to configure Cron to send Memory Metrics to CloudWatch every 5 minutes.

 

Verification completed successfully. No actual metrics sent to CloudWatch.

 

Step –4

Configure Crontab

 

Type crontab –e at the shell prompt and append below line to the end of the file.

 

*/5 * * * * /opt/aws-scripts-mon/mon-put-instance-data.pl --mem-util --from-cron

 

Note that the file would be empty if there are no existing cron jobs

 

 

Now save and exit.

 

After 10 or 15 minutes you would see Memory utilization of this instance in CloudWatch console.

 

Windows Instance

On your Windows Server EC2 Instance, download and install latest EC2Config utility

 

Goto - http://aws.amazon.com/developertools/5562082477397515

 

and click on download to get the latest Amazon Windows EC2Config Service

 

 

Extract the downloaded zip file.

 

 

Get into the extracted folder and double click to run the EXE

 

 

Follow onscreen instructions to complete the installation.

 

 

 

 

Now open the AWS.EC2.Windows.CloudWatch.json file in Notepad or Notepad++ with administrative privilege

 

Complete path of the file is

JSON Settings File

C:\Program Files\Amazon\Ec2ConfigService\Settings\AWS.EC2.Windows.CloudWatch.json

 

 

In AWS.EC2.Windows.CloudWatch.json file look for the section “Id”: “PerformanceCounter”

 

 

Modify the Parameters “DimensionName” and “DimensionValue” as shown in the screenshot with values “InstanceId” and “{instance_id}” respectively.

 

 

Now go to section “Id”: “CloudWatch”

 

 

Key in IAM user AccessKey and SecretKey at respective place holders as shown below

 

 

Change the Region to the AWS Region where the EC2 Instance is running. In my case it was Singapore – ap-southeast-1

 

Change NameSpace to “System/Windows” . By default this would be “Windows/Default”

Now go to Flows section of the file, Generally this would be at the End of File.

 

 

Replace "(ApplicationEventLog,SystemEventLog),CloudWatchLogs" with

"PerformanceCounter,CloudWatch"

 

 

Save and close the file.

 

Now go to start and look for EC2ConfigService Settings.

 

Windows 2008 R2

 

Windows 2012 R2

 

 

Check Enable CloudWatch Logs integration

 

 

Click on Apply and OK to save and close

 

Go to services and restart Ec2Config Service

 

 

After 10 to 15 minutes, you will be able to see the Memory Metrics in CloudWatch Console with default polling interval of 5 minutes.

 

 

You can monitor below log file for any errors.

 

Log File

C:\Program Files\Amazon\Ec2ConfigService\Logs\Ec2ConfigLog.txt

 

Commands for Instance Reference

These steps are almost similar on RHEL (RedHat Enterprise Linux) and Amazon Linux only slight difference with pre-requisite installation.

 

CEntOS 6.x Linux

yum -y install perl-DateTime perl-Sys-Syslog openssl openssl-devel cpan unzip

perl -MCPAN -e shell

install Bundle::LWP6 LWP

perl -MCPAN -e 'install LWP::Protocol::https'

 

Amazon Linux

yum -y install perl-DateTime perl-Sys-Syslog perl-LWP-Protocol-https cpan

perl -MCPAN -e shell

install Bundle::LWP6 LWP

 

RedHat Linux

yum -y install install perl-DateTime perl-Sys-Syslog zip unzip perl-DateTime perl-Sys-Syslog perl-LWP-Protocol-https perl-Digest-SHA

perl -MCPAN -e shell

install Bundle::LWP6 LWP

 

Ubuntu Linux

apt-get update
apt-get install unzip
apt-get install libwww-perl libdatetime-perl

 

Downloading and Configuring Scripts

wget http://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.1.zip

unzip CloudWatchMonitoringScripts-1.2.1.zip

rm -f CloudWatchMonitoringScripts-1.2.1.zip

mv aws-scripts-mon /opt/

cd /opt/aws-scripts-mon/

cp awscreds.template awscreds.conf

vi awscreds.conf

crontab -e

*/5 * * * * /opt/aws-scripts-mon/mon-put-instance-data.pl --mem-util --from-cron

 

Error-

If you ever get timezone.pm error, run below command.

 

yum install perl-DateTime-TimeZone

 

References and Download Links -

EC2 Linux Instance

http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/mon-scripts.html

Download - http://aws.amazon.com/code/8720044071969977

 

EC2 Windows Instance

https://blogs.aws.amazon.com/application-management/post/Tx1KG4IKXZ94QFK/Using-CloudWatch-Logs-with-Amazon-EC2-Running-Microsoft-Windows-Server

Download - http://aws.amazon.com/developertools/5562082477397515

 

Other References

http://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/ec2-configuration-cwl.html
http://www.tothenew.com/blog/setup-custom-cloudwatch-metrics-on-ec2-instance/
https://forums.aws.amazon.com/message.jspa?messageID=592245