[AWS] These are the helpful commands in AWS – awscli

Install aws cli

apt update -y
apt install unzip -y

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip > /dev/null 2>&1
sudo ./aws/install > /dev/null 2>&1

vì nếu chạy bằng script thì tạo nhiều log ở stdout nền bạn có thể sử dụng /dev/null 2>&1 ở cuối command.
https://askubuntu.com/questions/474556/hiding-output-of-a-command

No such file or directory: ‘less’: ‘less’

[Errno 2] No such file or directory: 'less': 'less'
Exited with code exit status 255

https://github.com/aws/aws-cli/issues/5038#issue-575848470

sudo apt-get update && sudo apt-get install -yy less

aws ….

Mình sẽ note command hay sài trên aws:

>>>>>
>>>>>
>>>>>

aws configure --profile <profile-name>

aws configure list-profiles

aws sts get-caller-identity --profile <profile-name>

aws eks --region <region_name> update-kubeconfig --name <eks_cluster_name> --profile <profile-name>

aws eks list-clusters --region <region_name> --profile <profile_name>

aws eks --region <region-code> update-kubeconfig --name <cluster_name>
aws eks --region us-east-1 update-kubeconfig --name SAP-dev-eksdemo

grep -rH "text" /folder

Security Group for Inbound Connection

#https://docs.aws.amazon.com/cli/latest/reference/ec2/describe-security-group-rules.html
aws ec2 describe-security-group-rules \
    --filter Name="group-id",Values="sg-0ead082eb96a4cfd8" \
    --profile dev-mdcl-nimtechnology-engines

aws ec2 describe-security-groups --group-ids sg-0ead082eb96a4cfd8 --profile dev-mdcl-nimtechnology-engines

aws ec2 authorize-security-group-ingress --group-id sg-0ead082eb96a4cfd8 --protocol tcp --port 443 --cidr 0.0.0.0/0 --profile dev-mdcl-nimtechnology-engines

####delete Rule in SecGroup
aws ec2 revoke-security-group-ingress --group-id sg-0ead082eb96a4cfd8 --security-group-rule-ids sgr-xxxxx --profile dev-mdcl-nimtechnology-engines --region us-west-2

AWS Configure Bash One Liner

https://stackoverflow.com/questions/34839449/aws-configure-bash-one-liner

aws configure set aws_access_key_id "AKIAI44QH8DHBEXAMPLE" --profile user2 \
&& aws configure set aws_secret_access_key "je7MtGbClwBF/2Zp9Utk/h3yCo8nvbEXAMPLEKEY" --profile user2 \
&& aws configure set region "us-east-1" --profile user2 \
&& aws configure set output "text" --profile user2

>>>>using environment
aws configure set aws_access_key_id "$AWS_ACCESS_KEY_ID" --profile user2 \
&& aws configure set aws_secret_access_key "$AWS_ACCESS_KEY_SECRET" --profile user2 \
&& aws configure set region "$AWS_REGION" --profile user2 \
&& aws configure set output "text" --profile user2

IAM

EC2

s3

https://github.com/rishabkumar7/cloud-cheat-sheets

>>>>> https://stackoverflow.com/questions/27932345/downloading-folders-from-aws-s3-cp-or-sync

Using aws s3 cp from the AWS Command-Line Interface (CLI) will require the --recursive parameter to copy multiple files.

aws s3 cp --recursive s3://myBucket/dir localdir

The aws s3 sync command will, by default, copy a whole directory. It will only copy new/modified files.

aws s3 sync s3://mybucket/dir localdir

Just experiment to get the result you want.

Documentation:

• cp command
• sync command

check folder size on S3

aws s3 ls s3://<bucket_name>/<folder_name>/ --recursive --summarize --human-readable

example: aws s3 ls s3://nim-dataset-of-rainmaker/avtest/ --recursive --summarize --human-readable

Checksum S3 files by Golang

https://github.com/aws-samples/amazon-s3-checksum-tool

S3 express one zone

copy file from s3 standard to s3 express one zone

root@LE11-D7891:~# time aws s3 cp s3://artifactory/1GB.txt s3://demo-s3-express-one-zone--usw2-az1--x-s3/1GB-v3.txt
copy: s3://artifactory/1GB.txt to s3://demo-s3-express-one-zone--usw2-az1--x-s3/1GB-v3.txt

real    0m32.645s
user    0m0.905s
sys     0m0.094s

copy file from s3 express one zone to s3 standard

time aws s3 cp s3://demo-s3-express-one-zone--usw2-az1--x-s3/1GB.txt s3://artifactory-nim/1GB-v5.txt --storage-class ONEZONE_IA --copy
-props none
copy: s3://demo-s3-express-one-zone--usw2-az1--x-s3/1GB.txt to s3://artifactory-nim/1GB-v5.txt

real    0m14.737s
user    0m0.872s
sys     0m0.087s

Secrets Manager.

https://www.learnaws.org/2022/08/28/aws-cli-secrets-manager/#how-to-list-all-secrets

ECR

Retagging an image: https://docs.aws.amazon.com/AmazonECR/latest/userguide/image-retag.html

You can retag without pulling or pushing the image with Docker.

Mình có tìm được 1 script của 1 anh trai:

Để chạy được script trên thì:

chmod +x ecr_add_tag.sh

Source the Script: You can source the script into your shell so that you can use the ecr-add-tag function directly:

source ecr_add_tag.sh

How to Use the Function
After sourcing the script, you can use the function ecr-add-tag in your terminal as follows:

ecr-add-tag ECR_REPO_NAME TAG_TO_FIND TAG_TO_ADD [AWS_PROFILE]

• ECR_REPO_NAME: The name of the ECR repository.
• TAG_TO_FIND: The existing tag of the image you want to re-tag.
• TAG_TO_ADD: The new tag you want to add to the image.
• [AWS_PROFILE]: (Optional) The AWS profile to use. If not provided, the default profile is used.

Example:

ecr-add-tag my-ecr-repo 1.0.0 1.0.1 my-aws-profile

Script to be easier

#!/usr/bin/env bash

# Disable AWS CLI pager
export AWS_PAGER=""

# Check for the correct number of arguments
if (( $# < 3 )); then
  echo "Wrong number of arguments. Usage: $0 ECR_REPO_NAME TAG_TO_FIND TAG_TO_ADD [AWS_PROFILE]"
  exit 1
fi

# Parse the arguments
repo_name=$1
existing_tag=$2
new_tag=$3
profile=$4

# If a profile is provided, format it correctly
[[ ! -z "$profile" ]] && profile="--profile ${profile}"

# Fetch the existing image manifest
manifest=$(aws ecr batch-get-image ${profile} \
                    --repository-name $repo_name \
                    --image-ids imageTag=$existing_tag \
                    --query 'images[].imageManifest' \
                    --output text)

# Add the new tag to the image
aws ecr put-image ${profile} \
                  --repository-name $repo_name \
                  --image-tag $new_tag \
                  --image-manifest "${manifest}"

Sau đó bạn run:

bash ecr-add-tag.sh alpine/terragrunt 1.1.7-eks 1.1.7-eks.v6 nim-dev

AWS CLi Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    workload.user.cattle.io/workloadselector: apps.deployment-nim-engines-dev-aws-cli
  name: aws-cli
  namespace: nim-engines-dev
spec:
  replicas: 1
  selector:
    matchLabels:
      workload.user.cattle.io/workloadselector: apps.deployment-nim-engines-dev-aws-cli
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      labels:
        workload.user.cattle.io/workloadselector: apps.deployment-nim-engines-dev-aws-cli
      namespace: nim-engines-dev
    spec:
      containers:
        - command:
            - /bin/sh
            - '-c'
            - while true; do sleep 3600; done
          env:
            - name: AWS_ACCESS_KEY_ID
              value: XXXXXXXXXXX
            - name: AWS_SECRET_ACCESS_KEY
              value: XXXXXXXXXXXXXXXL0eXPPXXXXHUU1bW
            - name: AWS_DEFAULT_REGION
              value: us-west-2
          image: amazon/aws-cli:latest
          imagePullPolicy: Always
          name: aws-cli
          resources: {}
          securityContext:
            allowPrivilegeEscalation: false
            privileged: false
            readOnlyRootFilesystem: false
            runAsNonRoot: false
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          volumeMounts:
            - mountPath: /app/downloaded
              name: file-service
      dnsPolicy: ClusterFirst
      nodeSelector:
        kubernetes.io/os: linux
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
      volumes:
        - name: file-service
          persistentVolumeClaim:
            claimName: pvc-file-service-smb-1