Monday, August 30, 2021

Kubectl Commands : Master List

kubectl get pods -n cs
kubectl get svc
kubectl get pods -o wide
kubectl exec -it <podName> -n fda-qa /bin/bash
kubectl describe <podName>
-------------------------------------
kubectl apply -f csv.yml
-------------------------------------
sudo az acr login --name coraaidev
-------------------------------------
kubectl logs <podName> -n cs
 kubctl delete pod <podName> -n cs
 -------------------------------------
sudo docker build -t dev.azurecr.io/hello-service  .
 sudo docker push dev.azurecr.io/hello-service
sudo docker images
-------------------------------------
sudo docker build -t association-service .
sudo docker tag association-service:latest coraaidev.azurecr.io/association-service:latest
docker tag -t  coraidev.azureecr.io/vea-cs:latest .
sudo docker push coraidev.azureecr.io/vea-cs:latest

AWS Notes - Useful

https://aws.amazon.com/premiumsupport/knowledge-center/manage-service-limits/

How do I manage my AWS service quotas?

For all other quota increase requests, create a case in the AWS Support console, and then choose Service Limit Increase.

-------------------------------------

https://spanning.com/blog/types-of-backup-understanding-full-differential-incremental-backup/

How Many Types of Backup are There?

There are mainly three types of backup: full, differential, and incremental.

Full - Entire Dataset regardless of any previous backups

Differential - Additions or alterations since most recent full backup

Incremental - Additions or alterations since most recent Incremental backup.

https://1tskcg39n5iu1jl9xp2ze2ma-wpengine.netdna-ssl.com/wp-content/uploads/2020/03/types-of-backup-full-differential-incremental.png

-------------------------------------

Amazon EBS volumes are placed in a specific Availability Zone where they are automatically replicated to protect you from the failure of a single component. 

All EBS volume types offer durable snapshot capabilities and are designed for 99.999% availability.

At no additional charge to you, Amazon EBS volume data is replicated across multiple servers in an Availability Zone to prevent the loss of data from the failure of any single component.

https://aws.amazon.com/ebs/features/

-------------------------------------

What Is the Difference Between AWS Snapshot and Backup?

https://www.nakivo.com/blog/difference-aws-snapshot-backup/


-------------------------------------

https://www.nagarro.com/en/blog/a-quick-guide-to-disaster-recovery-solution-with-aws

https://docs.aws.amazon.com/whitepapers/latest/disaster-recovery-workloads-on-aws/disaster-recovery-options-in-the-cloud.html

AWS offers 4 basic techniques for back-up and disaster recovery.,

Backup and Restore   $

Pilot Light   $$

Warm Standby $$$

Multisite $$$ (Almost Realtime)


Recovery Time Objective

Recovery Point Objective

-------------------------------------

https://www.bangkokpost.com/tech/1153473/aws-makes-play-through-new-services

Snowmobile - Exabytes level data 


With the new Snowmobile service, AWS said a person could ship one exabyte (an exabyte is 1,000 petabytes) of data to AWS in about six months -- a feat that would take 26 years over a 10Gbps dedicated connection.

-------------------------------------

https://medium.com/@gabanox/aws-data-replication-and-redundancy-with-managed-services-2e4d2a0fe98e

AWS Cross-Region Replication (CRR) 

S3 Replication and Redundancy with Managed Services in AWS

"Source Bucket" in diff region same Account

"Destination Bucket" - in diff  Region same account

Add a replication Configuration on "Source Bucket" and it asynchronously copies Objects with or without Tags to Another destination Bucket

-------------------------------------

https://www.msp360.com/resources/blog/geo-redundancy-comparison-amazon-s3-microsoft-azure-google-cloud/


Amazon S3 Geo-Redundancy and Replication

Amazon S3, the storage service on AWS, provides two types of geo-redundancy options.


S3 Availability Zones

The first method for achieving geo-redundancy on AWS is by using what AWS calls Availability Zones. Each AWS region includes multiple Availability Zones, which are geographically distant from each other.


By default, the Standard, Infrequent Access and Glacier storage classes on S3 replicate data automatically across at least three Availability Zones. This means that as long as you are using one of these S3 storage classes, your data will be geo-redundant, without any extra effort required on your part. The cost of this AWS geo-redundancy is built into the standard S3 pricing.


The major downside to achieving AWS geo-redundancy via this approach is that the Availability Zones within the same AWS region are still somewhat close together. Amazon is not specific about how close they are to each other, but says only that they are separated by “miles.” As such, they will suffice to protect against disruptions that are very localized, such as a fire or cooling system failure that affects just one server room. However, a large-scale disaster, such as major flooding or an earthquake, could impact multiple Availability Zones within the same region; in that case, your data may cease to be available despite the geo-redundancy that you have established via Availability Zones.


AWS Cross-Region Replication (CRR)

-------------------------------------

Amazon S3 Reduced Redundancy Storage

https://aws.amazon.com/s3/reduced-redundancy/

 

-------------------------------------

Durability

Standard                                                           99.999999999%

Standard - Infrequent Access 99.999999999%


Reduced Redundancy Storage 99.99%


-------------------------------------

8 Coding Challenges and Competitions That May Lead to Money or Jobs

https://www.makeuseof.com/tag/si-coding-challenges-lead-money-jobs/

-------------------------------------

https://aws.amazon.com/about-aws/global-infrastructure/?p=ngi&loc=0

-------------------------------------

https://aws.amazon.com/ec2/instance-types/

https://www.parkmycloud.com/blog/ec2-instance-types/

https://p2zk82o7hr3yb6ge7gzxx4ki-wpengine.netdna-ssl.com/wp-content/uploads/Screen-Shot-2020-02-21-at-1.29.51-PM-1536x992.png

-------------------------------------

https://aws.amazon.com/backup/

-------------------------------------

Amazon FSx

-------------------------------------

Amazon Data Lifecycle Manager

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/snapshot-lifecycle.html


-------------------------------------

Create an Amazon EBS-backed Linux AMI

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/creating-an-ami-ebs.html

-------------------------------------

SCP : Source Control Policies

Root Account : No resource here

Member Account : Children Accounts

Policies applied here those applied at Individual Account Level at AWS IAM

https://docs.aws.amazon.com/organizations/latest/userguide/orgs_getting-started_concepts.html

-------------------------------------

IAM user limit is 5000 per AWS account. 

https://stackoverflow.com/questions/28689364/aws-iam-users-limit-if-exceeds

The default maximum limit is 5000 users per AWS account. Beyond which you need to workout a solution with STS - http://docs.aws.amazon.com/STS/latest/UsingSTS/Welcome.html.

-------------------------------------

https://aws.amazon.com/fsx/windows/

Amazon FSx for Windows File Server provides fully managed, highly reliable, and scalable file storage that is accessible over the industry-standard Server Message Block (SMB) protocol. It is built on Windows Server,

-------------------------------------

Updating stacks using change sets

https://docs.amazonaws.cn/en_us/AWSCloudFormation/latest/UserGuide/using-cfn-updating-stacks-changesets.html

When you need to update a stack, understanding how your changes will affect running resources before you implement them can help you update stacks with confidence. Change sets allow you to preview how proposed changes to a stack might impact your running resources, for example, whether your changes will delete or replace any critical resources, Amazon CloudFormation makes the changes to your stack only when you decide to execute the change set, allowing you to decide whether to proceed with your proposed changes or explore other changes by creating another change set.


-------------------------------------

Choosing a routing policy for AWS Route 53

https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-policy.html

-------------------------------------

https://aws.amazon.com/s3/transfer-acceleration/

Amazon S3 Transfer Acceleration can speed up content transfers to and from Amazon S3 by as much as 50-500% for long-distance transfer of larger objects. 

-------------------------------------

Capacity Reservation 

Reserved Instances  -  Discounted Pricing - You commit USAGE

Savings Plan - Discounted Pricing - You commit SPEND


There are two classes of Reserved Instances:

Standard Reserved Instances lock you into using one instance family, on the same operating system.

Convertible Reserved Instances offer the flexibility to change families, operating systems and tenancy, but at the cost of a slightly smaller discount


https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-capacity-reservations.html

https://www.cloudhealthtech.com/blog/aws-reserved-instances-vs-on-demand

-------------------------------------

https://aws.amazon.com/professional-services/CAF/

AWS Cloud Adoption Framework

AWS Professional Services created the AWS Cloud Adoption Framework (AWS CAF) to help organizations develop and execute efficient and effective plans for their cloud adoption journey.

-------------------------------------

ECS vs Fargate

https://www.dragonspears.com/blog/aws-container-orchestration-101-ecs-vs-fargate-vs-eks

https://containersonaws.com/introduction/ec2-or-aws-fargate/

https://cloudonaut.io/ecs-vs-fargate-whats-the-difference/

https://cloudonaut.io/images/2019/02/ecs-container-instance-fargate@730w.webp

ECS --->  Uses EC2 or

Uses Fargate

-------------------------------------


EBS snapshots should be directly compared to replicas as they 


Replication

Replication creates a copy of your data which is stored in an identical format as the original

data. Because your replica is the same as your original machine, you can simply power on the

replica for instant disaster recoveryprovide similar functionality


Backup

A backup is a copy of your data that is stored in a special format in order to reduce the data

size and enable a full/granular recovery. Backups allow you to recover all or part of the data

from weeks, months, or even years ago. At the same time, backups take up a fraction of the

storage space when compared to the original data size


-------------------------------------

Saturday, August 21, 2021

Friday, August 20, 2021

AWS : Direct Connect

https://docs.aws.amazon.com/directconnect/latest/UserGuide/Welcome.html

Elastic IP Address : Delete /Disassociate

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/elastic-ip-addresses-eip.html


https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Scenario4.html


1st Disassociate Elastic IP Address

then Release Elastic IP Address


if Elastic IP is not bound, its chargeable

AWS Routing : 101

https://medium.com/@mda590/aws-routing-101-67879d23014d


https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Scenario4.html



https://karanawsbucket.s3.us-east-1.amazonaws.com/Documents/AWS%20Routing%20101.%20Introduction%20_%20by%20Matt%20Adorjan%20_%20Medium.pdf

User Data :AWS : 80 : WebPort

https://gist.github.com/nktstudios/fcaf50c30f4ce50790876a3fe5dd8ac7


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
#!/bin/bash

# update the machine
yum update -y

# install http
yum install -y httpd.x86_64

# start the service
systemctl start httpd.service

# enable service across reboots
systemctl enable httpd.service

# create the html page to render
echo "<h1>Hello World from NKT Studios!</h1>" >/var/www/html/index.html

AWS Certificate Manager : ACM : Public and Private CA Certificate

https://www.youtube.com/watch?v=bWPTq8z1vFY

https://docs.aws.amazon.com/acm/latest/userguide/gs-acm-request-public.html

https://gist.github.com/nktstudios/fcaf50c30f4ce50790876a3fe5dd8ac7

https://docs.aws.amazon.com/acm/latest/userguide/gs-acm-request-private.html

https://docs.aws.amazon.com/acm/latest/userguide/acm-overview.html

https://docs.aws.amazon.com/acm/latest/userguide/gs-acm-request-public.html

https://aws.amazon.com/certificate-manager/faqs/

https://docs.aws.amazon.com/acm/latest/userguide/acm-services.html

Thursday, August 19, 2021

AWS : Cloudwatch : Push EC2 Logs via "Cloudwatch Agent" to Cloudwatch

https://www.youtube.com/watch?v=ZCHwJLqPLj8

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/QuickStartEC2Instance.html

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AgentReference.html

Other Urls

https://cloudacademy.com/blog/centralized-log-management-with-aws-cloudwatch-part-1-of-3/

https://medium.com/tensult/to-send-linux-logs-to-aws-cloudwatch-17b3ea5f4863

https://www.strongdm.com/docs/installation/configure-logging/aws-cloudwatch


Edit    /etc/awslogs/awslog.conf

-----------------------------------------------------------------

[/var/log/messages]

datetime_format = %b %d %H:%M:%S

file = /var/log/messages

buffer_duration = 5000

log_stream_name = {instance_id}

initial_position = start_of_file

log_group_name = /var/log/messages


[/ActorLocatorAPI/logs]

datetime_format = %b %d %H:%M:%S

file = /tmp/logs/contoso-actorLocator.log

buffer_duration = 500

log_stream_name = {instance_id}

initial_position = start_of_file

log_group_name = /ActorLocatorAPI/logs

-----------------------------------------------------------------

You can send On Premise M/C as well with "Coudwatch Agent"


sudo yum install -y awslogs 


Edit file /etc/awslogs/awscli.conf    and change your AWS Region.

Edit file /etc/awslogs/awslogs.conf    and verify following lines                                                     Here You specify from where to read/What to read and push to Cloudwatch

AWS : CloudFormation

AWS CloudFormation Tutorial | AWS CloudFormation Demo | AWS Tutorial | AWS Training | Edureka - YouTube

Monday, August 9, 2021

kubectl : How to copy files from within Docker kubectl into PC/Laptop

//Inside Container
kubectl exec -it -n fda mongodb-2 /bin/bash
cd tmp
mkdir -p 10Aug2021
mongoexport --db fda-platform-extraction --collection cleanup_metadata --out    /tmp/10Aug2021/cleanup_metadata_DEV.json 

//Inside JumpServer
mkdir -p 10Aug2021
cd 10Aug2021
kubectl cp mongodb-2:/tmp/10Aug2021/  .  -n fda

NodeJs : npm run-script "sciptname" in package.json

npm run-script scriptName

This actually looks for "scriptName" in package.json  in "scripts" part of package.json
npm run-script build_dev


"name": "metronic-angular",
    "version": "6.1.8",
    "description": "Packages used by Angular",
    "scripts": {
        "ng": "ng",
        "start": "node --max_old_space_size=8192 node_modules/@angular/cli/bin/ng serve",
        "build": "ng build",
        "build_dev": "node --max_old_space_size=8192 node_modules/@angular/cli/bin/ng build --sourceMap=false --base-href=\"/\" --aot --output-hashing none",
        "build_qa": "node --max_old_space_size=8192 node_modules/@angular/cli/bin/ng build --configuration=qa --base-href=\"/\" --output-hashing none",
        "build_dev2": "node --max_old_space_size=8192 node_modules/@angular/cli/bin/ng build --configuration=dev2 --base-href=\"/\" --output-hashing none",
        "build_qa2": "node --max_old_space_size=8192 node_modules/@angular/cli/bin/ng build --configuration=qa2 --base-href=\"/\" --output-hashing none",
        "build_stage": "node --max_old_space_size=8192 node_modules/@angular/cli/bin/ng build --configuration=staging --base-href=\"/\" --output-hashing none",
        "build_prod": "node --max_old_space_size=8192 node_modules/@angular/cli/bin/ng build --prod --base-href=\"/\" --output-hashing none",
        "test": "ng test",
        "lint": "ng lint",
        "e2e": "ng e2e",
        "bundle-report": "webpack-bundle-analyzer dist/stats.json",
        "rtl": "webpack"
    },


Wednesday, August 4, 2021

Kubectl - Kubernetes -1

eks@ABCDAWS01:/$ kubectl get pods -n fda | awk '/case-management/{print $1}'
case-management-6b66d59f44-b6fph

eks@ABCDAWS01:/$ kubectl get pods -n fda | awk '/case-management/'
case-management-6b66d59f44-b6fph      1/1     Running   0          37m

eks@ABCDAWS01:/$ kubectl get pods -n fda |grep -E 'business|case|platform|output'
business-rules-68b8c55fd4-5gm8k       1/1     Running   0          37m
case-management-6b66d59f44-b6fph      1/1     Running   0          37m
output-generation-74b548d65b-fc44z    1/1     Running   0          37m
platform-7b5c6965c8-s2cv6             1/1     Running   0          36m

GetEnt : Linux Commands

getent command in Linux with examples

getent is a Linux command that helps the user to get the entries in a number of important text files called databases

https://www.geeksforgeeks.org/getent-command-in-linux-with-examples/

AWS : EC2 :Full Troubleshooting List

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/TroubleshootingInstancesConnecting.html#TroubleshootingInstancesServerError



Connection problems and errors

AWS : EC2: EBS : Mounting /Attaching EBS Volume : Duplicate UUID : Issue


sudo   mount -o   nouuid    /dev/xvdf2    /mnt/temp
 

https://stackoverflow.com/questions/55833917/unable-to-mount-a-volume-on-an-ec2-instance/55845557#55845557

https://stackoverflow.com/questions/55833917/unable-to-mount-a-volume-on-an-ec2-instance

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/TroubleshootingInstancesConnecting.html#TroubleshootingInstancesServerError

Linux "Stat" Comand | OCTAL Permission

 stat - -format="%a" /path1/subdir


https://www.cyberciti.biz/faq/get-octal-file-permissions-from-command-line-on-linuxunix/

https://unix.stackexchange.com/questions/188674/how-to-get-file-permission-in-octal


To just see octal file permissions on a GNU/Linux:
$ stat -c '%a' /etc/passwd

linux "Id" Command

https://www.geeksforgeeks.org/id-command-in-linux-with-examples/

Linux: List of all Users and All Groups

  • getent passwd
  • getent passwd | cut -d: -f1


  • less /etc/group
  • getent group
  • getent group | cut -d: -f1


Linux : /etc/passwd vs /usr/bin/passwd (passwd Utility) vs /etc/shadow vs /etc/group

https://stackoverflow.com/questions/50904342/etc-passwd-vs-usr-bin-passwd


The two files are different, and serve different purpose.

  • /etc/passwd is user database (fun fact: contrary to its name, it doesn't store passwords - those are stored (possibly in hashed form) in /etc/shadow) - see man 5 passwd (i.e. passwd(5)), man 5 shadow (i.e. shadow(5)).


  • /usr/bin/passwd is utility that is supposed to modify user records stored in /etc/passwd and /etc/shadow. See man 1 passwd (i.e. passwd(1))

Linux : File Hierarchy Standard : /bin vs /usr/bin vs usr/local/bin

https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard

https://unix.stackexchange.com/questions/5915/difference-between-bin-and-usr-bin

https://askubuntu.com/questions/308045/differences-between-bin-sbin-usr-bin-usr-sbin-usr-local-bin-usr-local

/sbin - Binaries needed for booting, low-level system repair, or maintenance (run level 1 or S)

/bin - Binaries needed for normal/standard system functioning at any run level.

/usr/bin - Application/distribution binaries meant to be accessed by locally logged in users

/usr/sbin - Application/distribution binaries that support or configure stuff in /sbin.

/usr/share/bin - Application/distribution binaries or scripts meant to be accesed via the web, i.e. Apache web applications

*local* - Binaries not part of a distribution; locally compiled or manually installed. There's usually never a /local/bin but always a /usr/local/bin and /usr/local/share/bin.



  • /bin : For binaries usable before the /usr partition is mounted. This is used for trivial binaries used in the very early boot stage or ones that you need to have available in booting single-user mode. Think of binaries like catls, etc.

  • /sbin : Same, but for binaries with superuser (root) privileges required.

  • /usr/bin : Same as first, but for general system-wide binaries.

  • /usr/sbin : Same as above, but for binaries with superuser (root) privileges required.

Linux : PAM : Pluggable Authentication Module

sudo pam_tally2 -u 703247699 --reset

Config Files for PAM
/etc/pam.d
/etc/security


sudo   pam_tally2   -u    userID123
---------------------------------------------------------
Login           Failures Latest failure     From
userID123    0





Linux - Special Dollar Sign Variables - $0, S1

https://stackoverflow.com/questions/5163144/what-are-the-special-dollar-sign-shell-variables

  • $1$2$3, ... are the positional parameters.
  • "$@" is an array-like construct of all positional parameters, {$1, $2, $3 ...}.
  • "$*" is the IFS expansion of all positional parameters, $1 $2 $3 ....
  • $# is the number of positional parameters.
  • $- current options set for the shell.
  • $$ pid of the current shell (not subshell).
  • $_ most recent parameter (or the abs path of the command to start the current shell immediately after startup).
  • $IFS is the (input) field separator.
  • $? is the most recent foreground pipeline exit status.
  • $! is the PID of the most recent background command.
  • $0 is the name of the shell or shell script.

Most of the above can be found under Special Parameters in the Bash Reference Manual. There are all the environment variables set by the shell.

For a comprehensive index, please see the Reference Manual Variable Index.


Azure - Pipeline - Add Approver for Stage

https://learn.microsoft.com/en-us/azure/devops/pipelines/process/approvals?view=azure-devops&tabs=check-pass