[S3] Try to compare S3 express One Zone with S3 standard.

S3 chắc chắn sẽ không còn xa lại với mọi người nhưng S3 express One Zone thì chắc chắn là xa lạ

S3 One Zone-IA (also known as Express One Zone) offers 99.999999999% (11 nines) of durability, but it only stores data in a single Availability Zone. This means if that AZ experiences an outage, your data could be temporarily or permanently lost.

S3 Standard: achieved by storing data redundantly across multiple Availability Zones (AZs) within a region. If one AZ goes down, your data will still be available.

S3 Standard replicates data across multiple AZs; S3 One Zone-IA stores it in only one AZ.
Vì thế tiền data transfer của S3 One Zone-IA thấp hơn S3 Standard

Compare S3 One Zone-IA to S3 Standard

Downloading test case.

Download a file via a Pre-signed URL

Bài test này bạn thực hiện tạo URL http để download 1 file trên S3,

Bài test này chúng ta work với file 200MB trên S3

S3 type	s3 standard	s3-express-one-zone
File Size	200MB
Download Type	Pre-signed URL
Max Speed	95.5 MiB/s	142 MiB/s
Min Speed	38.8 MiB/s	69.3 MiB/s

S3 Express-One-Zone is faster.

S3 Express-One-Zone offers 48.7% faster maximum download speeds and 78.6% faster minimum speeds than S3 Standard for this 200MB file size when using pre-signed URLs.

Download a file from S3 using the S3 SDK.

Ở bài test này chúng ta sẽ download file trược tiếp từ S3 bằng S3 SDK.

S3 type	s3 standard	s3-express-one-zone
File Size	200MB
Download Type	S3 SDK
Max Speed	94.7 MiB/s	141 MiB/s
Min Speed	77.7 MiB/s	134 MiB/s

S3 Express-One-Zone is faster

S3 Express-One-Zone offers 48.9% faster maximum download speeds and 72.5% faster minimum speeds than S3 Standard for this 200MB file size when using the S3 SDK.

Nếu compare chart Download Của S3 SDK và Pre-signed URL bạn sẽ thấy S3 SDK có tộc độ stable hơn.

S3 SDK and Trunk Size: 5MB

Ở đây chúng ta cần tìm hiểu 1 khái niệm là trunk file

Imagine an S3 “trunk file” as a large file that’s broken into smaller parts, making it easier to manage, transfer, or store. When dealing with huge files (like big videos or backups), S3 uses this method to make the process faster and more efficient.

S3 type	s3 standard	s3-express-one-zone
File Size	200MB
Download Type	S3 SDK and Trunk Size: 5MB
Max Speed	439 MiB/s	467 MiB/s
Min Speed	285 MiB/s	349 MiB/s

Thus, S3 Express-One-Zone consistently outperforms S3 Standard in both maximum and minimum speed metrics.
S3 Express-One-Zone offers 6.4% faster maximum download speeds and 22.5% faster minimum speeds than S3 Standard for this 200MB file size when using the S3 SDK and Trunk Size: 5MB.

Applying trunk size in the S3 SDK significantly improves download speed.

Copy a file from S3 standard to S3 One Zone IA.

root@LE11-D7891:~# bash measure_s3_speed.sh
copy: s3://artifactory/1GB.txt to s3://demo-s3-express-one-zone--usw2-az1--x-s3/1GB-v3.txt
File copy completed in 33 seconds
Transfer speed: 31.03 MB/s
root@LE11-D7891:~# bash measure_s3_speed.sh
copy: s3://artifactory/1GB.txt to s3://demo-s3-express-one-zone--usw2-az1--x-s3/1GB-v3.txt
File copy completed in 33 seconds
Transfer speed: 31.03 MB/s

Tốc độ copy file from S3 standard to S3 One Zone IA làm nim khá thất vọng: chỉ 31.03 MB/s.

Increase the pod count to test S3 One Zone IA.

S3 type		One Pod	10 Pods	30 Pods
File Size		1GB	1GB	1GB
Download Type: S3 SDK and Trunk Size: 5MB	Max Speed	812 MiB/s	872 MiB/s	802 MiB/s
	Min Speed	439 MiB/s	427 MiB/s	370 MiB/s

Bạn có thể thấy là khi tăng số lượng pod và kiểm tra tốc độ download thì bạn sẽ thấy chúng không thay đổi quá nhiều.

Issue

Trong qua trình integrate với S3 express One Zone nếu bạn gặp lỗi với Session:

Couldn't get testing files from s3 An error occurred (AccessDenied) when calling the CreateSession operation: Access Denied

Example bucket policies for directory buckets – Amazon Simple Storage Service

thì bạn cần thêm action là: s3express:CreateSession

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ReadWriteAccess",
            "Effect": "Allow",
            "Resource": "arn:aws:s3express:us-west-2:account-id:bucket/bucket-base-name--zone-id--x-s3",
            "Principal": {
                "AWS": [
                    "111122223333"
                ]
            },
            "Action": [
                "s3express:CreateSession"
            ]
        }
    ]
}