Sunday, March 06, 2016

Monitor EC2 Memory Usage using AWS CloudWatch

CloudWatch an AWS service provides rich set of tools to monitor health and resource utilization of various services on AWS. These metrics can be used to trigger an alarm, send notifications, even trigger actions based on these alarms.


One main metric that is missing from default CloudWatch metrics is Memory utilization of EC2 instances. Since AWS does not have access to EC2 instance at the Operating System level, only CPU, Network utilization, IO and other metrics that can be monitored through Hypervisor layer are available by default in AWS Console.


AWS provides a set of scripts that utilize CloudWatch API to send Memory Utilization info and make use of CloudWatch features to monitor and take action if alarms are raised.

The process of sending these custom metrics are different for Linux and Windows Instances. Even the process of installing pre-requisites on different Linux distributions is slightly different.


In this post we will see the process of installing and configuring CloudWatch Custom Metrics for Redhat derived Linux Distros, mainly RHEL, Amazon Linux and CentOS along with Windows to monitor Memory Utilization of the EC2 instance


Updated (27-March-2016) – Added Commands for Ubuntu Linux


List of all commands run on Linux are also available at the end of the post for easy



Step –1

Create an IAM user with access key. We do not need password or console access to this user.




Copy and keep the Access key handy as we would need this every time we configure custom metrics to be sent to CloudWatch




Create and Attach an Inline Policy to the user with below actions.



Policy would like below. You may copy paste this for your use.


  "Version": "2012-10-17",
  "Statement": [
      "Effect": "Allow",
      "Action": ["cloudwatch:PutMetricData", "cloudwatch:GetMetricStatistics", "cloudwatch:ListMetrics", "ec2:DescribeTags"],
      "Resource": "*"


Go to IAM Users, click on the user just created and click on “click here” hyperlink in Inline Policies




In next page select custom Policy and click select




Give a name for Policy and paste policy in Policy Document and click on Apply Policy




Now the user would have policy attached to send custom metrics using API to CloudWatch




You can verify the policy using Show Policy Actions in IAM console for the user.


Step –2

Installing pre-requisites, all the following commands are to be run with root privilege, you can use sudo as well, except inside CPAN shell.


CEntOS 6.x -


Command -1

yum -y install perl-DateTime perl-Sys-Syslog openssl openssl-devel cpan unzip


Command -2

perl -MCPAN -e shell


If this command prompts for input, accept defaults




The above command when complete would show CPAN prompt




Run below command at CPAN prompt


Command -3

install Bundle::LWP6 LWP


This would prompt for input multiple times, accept all default values or answer yes. This would take some time to complete, so, have patience.




One the command finishes successfully, the CPAN prompt number would change, type quit and hit enter to exit to shell prompt




Run below command at shell prompt and accept all defaults or answer yes


Command -4

perl -MCPAN -e 'install LWP::Protocol::https'


Once this finishes successfully, you will have shell prompt waiting for next steps.


Step –3

Downloading Scripts


Run below commands to download, extract, delete the downloaded zip file and move scripts to /opt folder, and open awscreds.conf file to configure credentials.




rm -f

mv aws-scripts-mon /opt/

cd /opt/aws-scripts-mon/

cp awscreds.template awscreds.conf

vi awscreds.conf




I have used vi for text editing, but you can use text editor of your choice.


Once awscreds.conf file is open in the text editor, key in Accesskey ID and Secret Key that we got when we created IAM user in Step-1




Save and exit text editor.


Now run below command to check if everything is OK.


/opt/aws-scripts-mon/ --mem-util --verify –verbose




If you see the output as below, everything is properly configured and we can proceed to configure Cron to send Memory Metrics to CloudWatch every 5 minutes.


Verification completed successfully. No actual metrics sent to CloudWatch.


Step –4

Configure Crontab


Type crontab –e at the shell prompt and append below line to the end of the file.


*/5 * * * * /opt/aws-scripts-mon/ --mem-util --from-cron


Note that the file would be empty if there are no existing cron jobs




Now save and exit.


After 10 or 15 minutes you would see Memory utilization of this instance in CloudWatch console.



Windows Instance

On your Windows Server EC2 Instance, download and install latest EC2Config utility


Goto -


and click on download to get the latest Amazon Windows EC2Config Service




Extract the downloaded zip file.




Get into the extracted folder and double click to run the EXE




Follow onscreen instructions to complete the installation.








Now open the AWS.EC2.Windows.CloudWatch.json file in Notepad or Notepad++ with administrative privilege


Complete path of the file is

JSON Settings File

C:\Program Files\Amazon\Ec2ConfigService\Settings\AWS.EC2.Windows.CloudWatch.json




In AWS.EC2.Windows.CloudWatch.json file look for the section “Id”: “PerformanceCounter”




Modify the Parameters “DimensionName” and “DimensionValue” as shown in the screenshot with values “InstanceId” and “{instance_id}” respectively.




Now go to section “Id”: “CloudWatch”




Key in IAM user AccessKey and SecretKey at respective place holders as shown below




Change the Region to the AWS Region where the EC2 Instance is running. In my case it was Singapore – ap-southeast-1


Change NameSpace to “System/Windows” . By default this would be “Windows/Default”

Now go to Flows section of the file, Generally this would be at the End of File.




Replace "(ApplicationEventLog,SystemEventLog),CloudWatchLogs" with





Save and close the file.


Now go to start and look for EC2ConfigService Settings.


Windows 2008 R2



Windows 2012 R2




Check Enable CloudWatch Logs integration




Click on Apply and OK to save and close


Go to services and restart Ec2Config Service




After 10 to 15 minutes, you will be able to see the Memory Metrics in CloudWatch Console with default polling interval of 5 minutes.




You can monitor below log file for any errors.


Log File

C:\Program Files\Amazon\Ec2ConfigService\Logs\Ec2ConfigLog.txt


Commands for Instance Reference

These steps are almost similar on RHEL (RedHat Enterprise Linux) and Amazon Linux only slight difference with pre-requisite installation.


CEntOS 6.x Linux

yum -y install perl-DateTime perl-Sys-Syslog openssl openssl-devel cpan unzip

perl -MCPAN -e shell

install Bundle::LWP6 LWP

perl -MCPAN -e 'install LWP::Protocol::https'


Amazon Linux

yum -y install perl-DateTime perl-Sys-Syslog perl-LWP-Protocol-https cpan

perl -MCPAN -e shell

install Bundle::LWP6 LWP


RedHat Linux

yum -y install install perl-DateTime perl-Sys-Syslog zip unzip perl-DateTime perl-Sys-Syslog perl-LWP-Protocol-https perl-Digest-SHA

perl -MCPAN -e shell

install Bundle::LWP6 LWP


Ubuntu Linux

apt-get update
apt-get install unzip
apt-get install libwww-perl libdatetime-perl


Downloading and Configuring Scripts



rm -f

mv aws-scripts-mon /opt/

cd /opt/aws-scripts-mon/

cp awscreds.template awscreds.conf

vi awscreds.conf

crontab -e

*/5 * * * * /opt/aws-scripts-mon/ --mem-util --from-cron



If you ever get error, run below command.


yum install perl-DateTime-TimeZone


References and Download Links -

EC2 Linux Instance

Download -


EC2 Windows Instance

Download -


Other References


  1. Now EC2Config version has been changed in Win Server 2012 R2 and new version doesn't show 'Enable Cloudwatch logs integration" option in EC2config properties , due to which memory metric is not getting reflected to cloudwatch console.
    Please help how i can get it worked.

    Thanks in advance...

  2. EC2Config does not come with Windows Server 2016. It says to use the EC2 Launch Powershell. How does one use this to monitor the memory?

  3. I have the same problem like Shubham and Craig.. Any news about it?

  4. I am adding memory as well as disk metrics on windows server 2012 R2, but for disk metrics, in the Ec2ConfigLog.txt i am getting error -

    [Error] Host not configured, no data will be uploaded to CloudWatch

  5. nice tutorial.

  6. A Detour: From security perspective,
    In Step 1, rather than creating a user, create a role with policy and attach it to ec2 instance.
    Then, in Step 3, no need to add credentials to awscreds.conf

  7. The configuration for monitoring windows machines results with "Available Memory". Is it possible to get "Memory Utilized"?

  8. Nice Tutorial, Thanks for sharing the steps

  9. "Enable Cloudwatch logs integration"not showing up
    Logs as below:

    I am getting the following warning in the log file:

    Warning: Unable to Publish to WMI. | System.Management.Instrumentation.WmiProviderInstallationException: Exception of type 'System.Management.Instrumentation.WMIInfraException' was thrown. at System.Management.Instrumentation.InstrumentationManager.Publish(Object value) at Ec2Config.LegacyConfiguration.LegacyConfigurator.PublishWmiInstance()

    Failed to fetch instance metadata with exception The remote server returned an error: (404) Not Found. Failed to get metadata/user-data The remote server returned an error: (404) Not Found.


Share what you feel