Amazon S3, Glacier, Lifecycle Rules, and Versioning

The following notes that may help negotiate some of the intricacies of Amazon S3 and Amazon Glacier terrane. The post assumes a basic understanding of Amazon user and bucket procedures.

First, operations in the Amazon S3 console are somewhat restricted; for example, objects can be deleted, but names cannot be changed and objects cannot be moved. However, renaming and moving objects is straightforward in 3rd party software such as Cloudberry S3 Explorer (which I endorse – see Amazon S3, Glacier, and Cloudfront User Interface Options).

Upload speeds to S3 and Glacier are SLOW, and take considerable bandwidth. I run my uploads at night, and pause them in the morning. The Sync Folders option in Cloudberry Explorer solves problems related to interruption, system re-boots etc. I’m a fan.

When an object is created in S3 and moved to Glacier, the object remains in the S3 console but is marked for Glacier; retrieval is initiated from the S3 console. Lifecycle rules are used to actually mark an S3 object for Glacier (and also to schedule transfers and deletions), and the actual rules can take several hours to enact.

Buckets can be created in the Glacier console, but they cannot be accessed there, nor can they be accessed in this case from the S3 console; programming or third-party software is required. However, files can be moved between Glacier and your computer, using  for example FastGlacier or Cloudberry Explorer.

Whether a bucket is created in Glacier or moved there from S3, the contents may not be visible for immediate viewing, but only after an inventory request, that can take several hours to perform.

Versioning prevents the deletion of objects, by creating a new version every time an action is taken on a file. For example, a new version is created if a file is deleted or if a file is overwritten during a copy operation. The versions are visible in the S3 console (see the Show Versions tab); in Cloudberry, enable viewing of versions in the Tools > Options menu, and then ‘view versions’ in the file directory.

Versioning enables the restoration of earlier versions of an object (right click to initiate a restore), although the initiate restore command was not immediately accessible in the S3 console (?). Lifecycle rules can be applied to both current objects or previous versions of an object.

Related: Amazon S3-tagged posts

Image credit: http://tbt.no