AWS : Cloudwatch : Push EC2 Logs via "Cloudwatch Agent" to Cloudwatch

https://www.youtube.com/watch?v=ZCHwJLqPLj8

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/QuickStartEC2Instance.html

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AgentReference.html

Other Urls

https://cloudacademy.com/blog/centralized-log-management-with-aws-cloudwatch-part-1-of-3/

https://medium.com/tensult/to-send-linux-logs-to-aws-cloudwatch-17b3ea5f4863

https://www.strongdm.com/docs/installation/configure-logging/aws-cloudwatch

Edit    /etc/awslogs/awslog.conf

-----------------------------------------------------------------

[/var/log/messages]

datetime_format = %b %d %H:%M:%S

file = /var/log/messages

buffer_duration = 5000

log_stream_name = {instance_id}

initial_position = start_of_file

log_group_name = /var/log/messages

[/ActorLocatorAPI/logs]

datetime_format = %b %d %H:%M:%S

file = /tmp/logs/contoso-actorLocator.log

buffer_duration = 500

log_stream_name = {instance_id}

initial_position = start_of_file

log_group_name = /ActorLocatorAPI/logs

-----------------------------------------------------------------

You can send On Premise M/C as well with "Coudwatch Agent"

sudo yum install -y awslogs 

Edit file /etc/awslogs/awscli.conf    and change your AWS Region.

Edit file /etc/awslogs/awslogs.conf    and verify following lines                                                     Here You specify from where to read/What to read and push to Cloudwatch

AWS: EC2: Create Signature from Private Key - EC2 to verify if its the same one

openssl pkcs8 -in ./ProjKey.pem -inform PEM -outform DER -topk8 -nocrypt | openssl sha1 -c

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html#verify-key-pair-fingerprints

AWS : EC2 :Full Troubleshooting List

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/TroubleshootingInstancesConnecting.html#TroubleshootingInstancesServerError

Connection problems and errors

• Common causes for connection issues
• Error connecting to your instance: Connection timed out
• Error: unable to load key ... Expecting: ANY PRIVATE KEY
• Error: User key not recognized by server
• Error: Permission denied or connection closed by [instance] port 22
• Error: Unprotected private key file
• Error: Private key must begin with "-----BEGIN RSA PRIVATE KEY-----" and end with "-----END RSA PRIVATE KEY-----"
• Error: Server refused our key or No supported authentication methods available
• Cannot ping instance
• Error: Server unexpectedly closed network connection
• Error: Host key validation failed for EC2 Instance Connect

AWS:CloudShell:CLI: aws ec2 describe-instances

https://thehftguy.com/2016/03/10/how-to-export-amazon-ec2-instances-to-a-csv-file/

https://gmusumeci.medium.com/how-to-export-aws-ec2-instances-in-multiple-aws-regions-and-multiple-aws-accounts-to-excel-csv-ce283af0ed90

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Filtering.html

https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instances.html

https://docs.aws.amazon.com/cloudshell/latest/userguide/working-with-cloudshell.html

AWS Cloud Shell 

aws ec2 describe-instances --filters "Name=tag:Environment,Values=QA"  --output json

aws ec2 describe-instances --filters "Name=tag:Environment,Values=QA" --output table  >  QA_EC2_Instances.tsv

InstanceId

InstanceType

PrivateIpAddress

aws ec2 describe-instances \

--filters "Name=tag:Environment,Values=QA" \

--query 'Reservations[*].Instances[*].{InstanceId:InstanceId,InstanceType:InstanceType,PrivateIpAddress:PrivateIpAddress}' \

--output json \

>  QA_EC2_Instances.json

aws ec2 describe-instances --filters Name=instance-state-name,Values=running --query "Reservations[*].Instances[*].InstanceId" --output text

aws iam list-access-keys --user-name  john_doe

aws iam list-access-keys --user-name  john_doe

AWS : Mount EFS on EC2 - Part II - V Important

 sudo mkdir -p /efs

sudo chmod -R ugo+rwx /efs     

ls -ld /efs 

sudo vi /etc/fstab

fs-face524e.efs.us-east-1.amazonaws.com:/ /efs nfs4 rw,intr,hard,_netdev,nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport

sudo mount -a

sudo reboot now

chmod -R 777 /efs  is not safer

chmod -R ugo+rwx /efs is safer - Does not override SETUID AND SETGID BITS - Just Appends permissions

https://unix.stackexchange.com/questions/296675/is-chmod-r-ugorwx-safer-than-chmod-r-777

cd / && sudo umount /efs && sudo mv /efs /fileshare && sudo chmod ugo+w /fileshare  && cat /etc/fstab 

sudo sed  -i 's+/efs+/fileshare+g' /etc/fstab 

sudo mount -a && sudo reboot now

ls -l /fileshare 

df -h

ls -l /fileshare ; df -h

ls -l /fileshare && df -h

umount /efs 

sudo mv /efs /fileshare

sudo chmod ugo+w /fileshare

sudo vi /etc/fstab

sudo reboot now

https://www.cyberciti.biz/faq/how-to-use-sed-to-find-and-replace-text-in-files-in-linux-unix-shell/

sudo sed   's/efs/fileshare/g' /etc/fstab   //Replace content and its temporary

sudo sed   's+/efs+/fileshare+g' /etc/fstab  // Change Delimiter

sudo sed  -i 's+/efs+/fileshare+g' /etc/fstab //-i save changes to file/stream

sudo sed  -e '/fs-face524e/s/fileshare/helloooo/' /etc/fstab //Replace only if lines contain specific string "fs-face524e"

AWS : EFS : Mount Target and Mount in EC2

sudo apt-get -y install nfs-common
sudo yum -y install nfs-utils
sudo service nfs start
sudo service nfs status

sudo su -
sudo mkdir -p /efs
sudo chmod -R ugo+rwx /efs     
ls -ld /efs 

echo 'fs-face524e.efs.us-east-1.amazonaws.com:/ /efs nfs4 rw,intr,hard,_netdev,nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport 0 0' >> /etc/fstab

sudo mount -a

sudo reboot now

umount -f efs
sudo mount -a

#fs-face524e.efs.us-east-1.amazonaws.com:/ /efs nfs4 rw,intr,hard,_netdev,nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport 0 0

#echo 'fs-face524e.efs.us-east-1.amazonaws.com:/ /efs nfs4 rw,intr,hard,_netdev,nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport 0 0' | sudo tee -a /etc/fstab

#sudo vi /etc/fstab

chmod -R 777 /efs  is not safer

chmod -R ugo+rwx /efs is safer - Does not override SETUID AND SETGID BITS - Just Appends permissions

https://unix.stackexchange.com/questions/296675/is-chmod-r-ugorwx-safer-than-chmod-r-777

https://www.thegeekdiary.com/understanding-the-configuration-file-for-mounting-file-systems-etc-fstab/

https://askubuntu.com/questions/9939/what-do-the-last-two-fields-in-fstab-mean

The EFS mount helper is part of the amazon-efs-utils package. 

The amazon-efs-utils package is an open-source collection of Amazon EFS tools. 

For more information, see Manually installing the Amazon EFS client.

https://docs.aws.amazon.com/efs/latest/ug/installing-amazon-efs-utils.html#installing-efs-utils-amzn-linux

https://docs.aws.amazon.com/efs/latest/ug/installing-amazon-efs-utils.html#installing-other-distro

Before the Amazon EFS mount helper was available, 

we recommended mounting your Amazon EFS file systems using the standard Linux NFS client.

https://docs.aws.amazon.com/efs/latest/ug/mounting-fs-old.html#mounting-fs-install-nfsclient

https://docs.aws.amazon.com/efs/latest/ug/mounting-fs-mount-cmd-dns-name.html

https://docs.aws.amazon.com/efs/latest/ug/mounting-fs-mount-cmd-ip-addr.html

https://docs.aws.amazon.com/efs/latest/ug/efs-mount-helper.html#mount-fs-auto-mount-onreboot

To view and copy the exact commands to mount your EFS file system using the mount target IP address

Open the Amazon Elastic File System console at https://console.aws.amazon.com/efs/.

In the Amazon EFS console, choose the file system that you want to mount to display its details page.

To display the mount commands to use for this file system, choose Attach in the upper right.

The Attach screen displays the exact commands to use for mounting the file system.

https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html

https://kichik.com/2020/09/08/how-does-ec2-instance-profile-work/

https://computingforgeeks.com/mount-aws-efs-file-system-on-ec2/

https://docs.aws.amazon.com/efs/latest/ug/efs-mount-helper.html#mounting-access-points

sudo mkdir /efs

sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport fs-face524e.efs.us-east-1.amazonaws.com:/ efs

fs-face524e.efs.us-east-1.amazonaws.com:/ efs nfs4 defaults,_netdev 0 0

umount -f efs

sudo mount -a

Mounting on Amazon EC2 with a DNS name

Mounting with an IP address [https://docs.aws.amazon.com/efs/latest/ug/mounting-fs-mount-cmd-dns-name.html]

Mounting your Amazon EFS file system automatically[https://docs.aws.amazon.com/efs/latest/ug/mounting-fs-mount-cmd-ip-addr.html]

[https://docs.aws.amazon.com/efs/latest/ug/efs-mount-helper.html#mount-fs-auto-mount-onreboot]

NFS client

nfs-utils for RHEL, CentOS, Amazon Linux, and Fedora distributions

nfs-common for Debian and Ubuntu distributions

Network File Sharing (NFS) is a protocol that allows you to share directories and files with other Linux clients over a network

AWS : EBS Volumes : Attach and Mount on EC2

https://devopscube.com/mount-ebs-volume-ec2-instance/

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-using-volumes.html

• lsblk
• lsblk -f
• df -h
• less /etc/fstab

[ec2-user@ip-10-79-196-74 ~]$ lsblk

NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT

nvme0n1     259:0    0  150G  0 disk

├─nvme0n1p1 259:1    0    1M  0 part

└─nvme0n1p2 259:2    0  150G  0 part /

[ec2-user@ip-10-79-196-74 ~]$ lsblk -f

NAME        FSTYPE LABEL UUID                                 MOUNTPOINT

nvme0n1

├─nvme0n1p1

└─nvme0n1p2 xfs          77f1de26-38e6-4e1d-8a1e-baa1610669e6 /

[ec2-user@ip-10-79-196-74 ~]$ cat /etc/fstab

#

# /etc/fstab

# Created by anaconda on Mon Oct 28 17:51:10 2019

#

# Accessible filesystems, by reference, are maintained under '/dev/disk'

# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info

#

UUID=77f1de26-38e6-4e1d-8a1e-baa1610669e6 /                       xfs     defaults        0 0

[ec2-user@ip-10-79-196-74 ~]$ df -h

Filesystem      Size  Used Avail Use% Mounted on

devtmpfs         16G     0   16G   0% /dev

tmpfs            16G     0   16G   0% /dev/shm

tmpfs            16G   25M   16G   1% /run

tmpfs            16G     0   16G   0% /sys/fs/cgroup

/dev/nvme0n1p2  150G  6.5G  144G   5% /

tmpfs           3.1G     0  3.1G   0% /run/user/1000

[ec2-user@ip-10-79-196-74 ~]$

Mount EBS to EC2

https://devopscube.com/mount-ebs-volume-ec2-instance/

sudo cp /etc/fstab /etc/fstab.bak

/dev/xvdf       /hdd2   ext4    defaults,nofail        0       0