Skip to content

NimTechnology

Trình bày các công nghệ CLOUD một cách dễ hiểu.

  • Kubernetes & Container
    • Docker
    • Kubernetes
      • Ingress
      • Pod
    • Helm Chart
    • OAuth2 Proxy
    • Isito-EnvoyFilter
    • Apache Kafka
      • Kafka
      • Kafka Connect
      • Lenses
    • Vault
    • Longhorn – Storage
    • VictoriaMetrics
    • MetalLB
    • Kong Gateway
  • CI/CD
    • ArgoCD
    • ArgoWorkflows
    • Argo Events
    • Spinnaker
    • Jenkins
    • Harbor
    • TeamCity
    • Git
      • Bitbucket
  • Coding
    • DevSecOps
    • Terraform
      • GCP – Google Cloud
      • AWS – Amazon Web Service
      • Azure Cloud
    • Golang
    • Laravel
    • Python
    • Jquery & JavaScript
    • Selenium
  • Log, Monitor & Tracing
    • DataDog
    • Prometheus
    • Grafana
    • ELK
      • Kibana
      • Logstash
  • BareMetal
    • NextCloud
  • Toggle search form

[S3] You will naturally see files on S3 being deleted

Posted on April 15, 2024April 16, 2024 By nim No Comments on [S3] You will naturally see files on S3 being deleted

Bông nhiên bạn được đồng đội hỏi:

Chúng ta sẽ giả định là không có human nào động tay trong việc này:

Vậy là đã có cron nào đang mark detele tag vào file.

Ở đây chúng ta thấy có 1 lifecycle cho việc gì đó cùng tiềm hiểu nào

Sau khi mình edit chúng thì thấy 1 số configurations:

chúng ta sẽ cùng tìm hiểu Lifecycle rule actions

  1. Move current versions of objects between storage classes: This action allows you to automatically transition the latest version of stored objects to different storage classes. For example, you might move data from the standard storage class to a more cost-effective class like S3 Infrequent Access after it has not been accessed for a certain period.
  2. Move noncurrent versions of objects between storage classes: If you have versioning enabled on your S3 bucket, noncurrent (older) versions of an object can also be moved automatically to different storage classes. This helps in managing costs effectively, especially for data that needs to be retained but not frequently accessed.
  3. Expire current versions of objects: This action automatically deletes the current version of objects after a specified period. This is useful for data that only needs to be stored temporarily, such as log files or temporary backup data.
  4. Permanently delete noncurrent versions of objects: For buckets with versioning enabled, this rule allows you to automatically delete older versions of objects that are no longer needed, which helps in managing storage costs and maintaining compliance with data retention policies.
  5. Delete expired object delete markers or incomplete multipart uploads: This action cleans up expired object delete markers (which are placeholders left after the deletion of the latest version of a versioned object) and parts of incomplete multipart uploads that can accumulate and incur costs if not managed.

Ở bài viết này bạn có thể tìm hiểu về storage classes của S3:
https://bluexp.netapp.com/blog/aws-cvo-blg-s3-lifecycle-rules-using-bucket-lifecycle-configurations

Amazon S3 storage tier transitions can follow the waterfall model as illustrated below:

Sau khi đọc: Expire current versions of objects và Permanently delete noncurrent versions of objects sẽ gây ra vấn đề delete file.

chúng ta cùng tìm hiểu: Delete expired object delete markers or incomplete multipart uploads
In Amazon S3, the lifecycle rule action “Delete expired object delete markers or incomplete multipart uploads” doesn’t actually delete whole files but instead cleans up specific remnants that can clutter your storage:

  1. Expired Object Delete Markers: When you delete a file in a versioned bucket on S3, a “delete marker” is created. This marker is just a placeholder that tells S3 to treat the file as deleted. However, the actual versions of the file still exist until they are specifically deleted. The lifecycle action to delete expired object delete markers targets these placeholders. When you delete these markers, it does not affect the actual files or versions that may still be stored.
  2. Incomplete Multipart Uploads: If you start uploading a large file in parts but don’t finish, those unfinished parts remain in your storage. These parts are not useful because they don’t make up a complete file. The lifecycle rule to delete these incomplete uploads cleans up these leftover parts to save space and reduce costs.



AWS - Amazon Web Service

Post navigation

Previous Post: [Terraform Issue] Blocks of type “precondition” are not expected here, or Optional object type attributes are experimental.
Next Post: Gỡ lỗi chia sẻ link URL qua zalo, facebook…

More Related Articles

[Github-Action] Let’s use GitHub Action to build and push the docker image to the docker hub. AWS - Amazon Web Service
[AWS] Demo “code build” with experiment easily on AWS AWS - Amazon Web Service
[ElastiCache] Provisioning Redis on AWS so quickly by terraform AWS - Amazon Web Service
[EKS] operation error EKS: CreateAccessEntry, https response error StatusCode: 409 AWS - Amazon Web Service
[Security Group/Terraform] Look into some exciting in Security Group Terraform AWS - Amazon Web Service
[Monitoring] How to monitor EBS on AWS via Prometheus AWS - Amazon Web Service

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Tham Gia Group DevOps nhé!
Để Nim có nhiều động lực ra nhiều bài viết.
Để nhận được những thông báo mới nhất.

Recent Posts

  • [Argo Workflow] Create an access token for Argo Workflows July 14, 2025
  • [Argo Workflow] SSO Authentication for Argo Workflows. July 14, 2025
  • [AWS/EKS] Cache Docker image to accelerate EKS container deployment. July 10, 2025
  • [Laravel] Laravel Helpful June 26, 2025
  • [VScode] Hướng dẫn điều chỉnh font cho terminal June 20, 2025

Archives

  • July 2025
  • June 2025
  • May 2025
  • April 2025
  • March 2025
  • February 2025
  • January 2025
  • December 2024
  • November 2024
  • October 2024
  • September 2024
  • August 2024
  • July 2024
  • June 2024
  • May 2024
  • April 2024
  • March 2024
  • February 2024
  • January 2024
  • December 2023
  • November 2023
  • October 2023
  • September 2023
  • August 2023
  • July 2023
  • June 2023
  • May 2023
  • April 2023
  • March 2023
  • February 2023
  • January 2023
  • December 2022
  • November 2022
  • October 2022
  • September 2022
  • August 2022
  • July 2022
  • June 2022
  • May 2022
  • April 2022
  • March 2022
  • February 2022
  • January 2022
  • December 2021
  • November 2021
  • October 2021
  • September 2021
  • August 2021
  • July 2021
  • June 2021

Categories

  • BareMetal
    • NextCloud
  • CI/CD
    • Argo Events
    • ArgoCD
    • ArgoWorkflows
    • Git
      • Bitbucket
    • Harbor
    • Jenkins
    • Spinnaker
    • TeamCity
  • Coding
    • DevSecOps
    • Golang
    • Jquery & JavaScript
    • Laravel
    • NextJS 14 & ReactJS & Type Script
    • Python
    • Selenium
    • Terraform
      • AWS – Amazon Web Service
      • Azure Cloud
      • GCP – Google Cloud
  • Kubernetes & Container
    • Apache Kafka
      • Kafka
      • Kafka Connect
      • Lenses
    • Docker
    • Helm Chart
    • Isito-EnvoyFilter
    • Kong Gateway
    • Kubernetes
      • Ingress
      • Pod
    • Longhorn – Storage
    • MetalLB
    • OAuth2 Proxy
    • Vault
    • VictoriaMetrics
  • Log, Monitor & Tracing
    • DataDog
    • ELK
      • Kibana
      • Logstash
    • Fluent
    • Grafana
    • Prometheus
  • Uncategorized
  • Admin

Copyright © 2025 NimTechnology.