[EKS/Pods] Why can not the pod on EKS call http://169.254.169.254/latest/api/token

Sau khi mày mò 1 thứ trên argo workflow mình phát hiện là.
Nếu pod muốn lấy tocken của EC2 (Con node mà đang chưa pod đó) thì nó sẽ call vào http://169.254.169.254/latest/api/token.

NHưng thường sẽ bị timout.

time="2023-05-12T17:41:01.548Z" level=info msg="Check if directory" artifactName=main-logs duration=1.013138345s error="Put \"http://169.254.169.254/latest/api/token\": context deadline exceeded" key=artifacts-7fx8p/artifacts-7fx8p/main.log
time="2023-05-12T17:41:01.548Z" level=error msg="Artifact Server returned internal error" error="Put \"http://169.254.169.254/latest/api/token\": context deadline exceeded"
time="2023-05-12T17:41:01.548Z" level=info duration=1.017800207s method=GET path=/artifact-files/argo-workflow/workflows/artifacts-7fx8p/artifacts-7fx8p/outputs/main-logs size=22 status=500time="2023-05-12T17:41:06.208Z" level=info duration="163.742µs" method=GET path=index.html size=473 status=0

Vậy thì chúng ta increase the max allowed hops to 2

Bạn tưởng tượng là nếu node gọi đến 169.254.169.254 chị qua 1 HOP
Còn nếu pod call đến api 169.254.169.254 sẽ mất 2 hops.

https://github.com/weaveworks/eksctl/issues/1979#issuecomment-629962708

Nếu bạn muốn pod call được:

aws ec2 modify-instance-metadata-options \
    --instance-id i-090240284d9c5d5ee \
    --http-put-response-hop-limit 2 \
    --http-endpoint enabled \
    --profile nim-staging \
    --region us-west-2

Contents

What will EC2 use “http://169.254.169.254/latest/api/token” to do?

The URL “http://169.254.169.254/latest/api/token” is used by an Amazon EC2 instance to retrieve a token for use with the Instance Metadata Service version 2 (IMDSv2).

IMDSv2 was introduced by Amazon to provide a more secure way for EC2 instances to access their metadata. Unlike the previous version (IMDSv1), which did not require any form of authentication, IMDSv2 uses session-oriented requests. This means that before any data can be retrieved, a session must first be created by obtaining a token.

The EC2 instance would make a PUT request to “http://169.254.169.254/latest/api/token” to create a session and retrieve a token. This token is then used in all subsequent GET requests to the metadata service.

Here is an example of how to do this using curl:

TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`

This command will create a token that is valid for 6 hours (21600 seconds). This token can then be used in a GET request to retrieve metadata:

curl -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/

Using a token in this way makes it more difficult for unauthorized users or processes to access the metadata service, which can contain sensitive information about the EC2 instance. This is particularly important in preventing certain types of attacks, such as Server Side Request Forgery (SSRF).

Server Side Request Forgery (SSRF)

Server Side Request Forgery (SSRF) is a type of attack where an attacker tricks a server into making a request that it did not intend to make. This is often done by manipulating a URL that the server will request.

In the context of an Amazon EC2 instance, imagine that you have a web application running on your EC2 instance. This application has a feature that allows it to fetch an image from a URL, and the URL is supplied by the user.

Here’s how an SSRF attack might happen:

An attacker learns that your application is running on an Amazon EC2 instance.
They know that “http://169.254.169.254/latest/meta-data/” is the URL that EC2 instances use to retrieve their metadata, which can contain sensitive data.
Instead of providing the URL of an image, the attacker provides the URL of the EC2 metadata service.
If your application does not validate the URL properly, it will make a request to the metadata service and return the result to the attacker, exposing sensitive data.

With the introduction of Instance Metadata Service version 2 (IMDSv2), Amazon introduced a token-based mechanism to access the metadata. Now, to access the metadata, the application must first make a PUT request to “http://169.254.169.254/latest/api/token” to get a token, and then use that token in the GET request to access the metadata.

So, if the attacker tries the same trick in the IMDSv2 environment, the application will not be able to access the metadata just by providing the metadata service URL. The application would first need to make a PUT request to get a token, which is a step that the attacker cannot perform simply by providing a URL.

This is how IMDSv2 helps to protect against SSRF attacks.