What Is Amazon S3? Buckets, Object Storage and Access for Backends

Amazon Simple Storage Service (S3) is AWS's object storage: files of any size live as objects inside buckets, addressed by key rather than by a filesystem path. It is where AWS backends put user uploads, exports, media, backups and static assets.

Creating a bucket takes a minute in the console, and the decisions that matter for a backend are the Region, the storage class, how services and browsers get access, and how each environment keeps its own bucket without copying credentials around. This guide covers those pieces and how Encore (infrastructure from code) provisions S3 from code.

Amazon S3 at a glance

How S3 works

You create a general purpose bucket in an AWS Region, then upload objects. Each object has a key that uniquely identifies it inside the bucket. The combination of bucket, key and optional version ID addresses the object. S3 provides strong read-after-write consistency for PUTs and DELETEs in all Regions, so a successful upload is readable immediately.

Storage classes trade storage price for retrieval price. S3 Standard suits data the application reads regularly. S3 Standard-IA and S3 One Zone-IA cost less to store and more to read. S3 Glacier Instant Retrieval, Flexible Retrieval and Deep Archive suit long-lived archives. S3 Intelligent-Tiering moves objects between tiers as access patterns change. Lifecycle rules transition or expire objects by age.

S3 Versioning keeps prior versions when an object is overwritten or deleted, which helps recover from mistakes. Every new object upload is encrypted at rest with SSE-S3 by default; you can switch the bucket default to SSE-KMS when you need customer-managed keys and audit trails in KMS.

Access control and signed URLs

Buckets and objects are private by default. Block Public Access is on for new buckets, and modern setups disable ACLs so access is managed with IAM and bucket policies. A backend on ECS or EKS reaches S3 through its task or pod IAM role. That role should allow only the actions and bucket ARNs the service needs.

Browsers should not receive those credentials, so the usual pattern is a presigned URL: the backend signs a short-lived URL for one key, and the browser uploads or downloads directly against S3, which keeps large files off the API servers. Public assets can live in a public bucket, usually behind CloudFront.

What a hand-built setup involves

A typical S3 setup for a backend has a bucket per environment with a unique name, Block Public Access settings, a bucket policy or IAM grants for each service role, CORS for browser uploads, optional versioning and lifecycle rules, an environment variable carrying the bucket name, and a local path such as MinIO or LocalStack for development. In Terraform that is several resources per bucket, and every new bucket repeats them.

How Encore provisions S3

Encore combines an open source infrastructure SDK for TypeScript and Go with a platform that provisions and deploys into your own AWS or GCP account. You declare a bucket in code, and on AWS Encore creates a dedicated S3 bucket with a unique name, provisions an IAM role for the service, and serves public buckets through CloudFront, as listed in the AWS infrastructure docs. On GCP the same declaration becomes a Cloud Storage bucket.

Set up the app

A Go service that streams an upload into the bucket and hands out a short-lived download link:

// avatars/avatars.go
package avatars

import (
	"context"
	"io"
	"net/http"
	"time"

	"encore.dev"
	"encore.dev/beta/errs"
	"encore.dev/storage/objects"
)

// An S3 bucket on AWS, a Cloud Storage bucket on GCP.
var avatars = objects.NewBucket("avatars", objects.BucketConfig{})

const maxSize = 5 << 20 // 5 MB

//encore:api public raw method=PUT path=/avatars/:userID
func Upload(w http.ResponseWriter, req *http.Request) {
	userID := encore.CurrentRequest().PathParams.Get("userID")
	ctype := req.Header.Get("Content-Type")
	if ctype != "image/png" && ctype != "image/jpeg" {
		http.Error(w, "only png and jpeg", http.StatusUnsupportedMediaType)
		return
	}

	up := avatars.Upload(req.Context(), "users/"+userID, objects.WithUploadAttrs(objects.UploadAttrs{
		ContentType: ctype,
	}))
	_, err := io.Copy(up, http.MaxBytesReader(w, req.Body, maxSize))
	if err != nil {
		up.Abort(err)
		http.Error(w, "upload failed", http.StatusBadRequest)
		return
	}
	if err := up.Close(); err != nil {
		http.Error(w, "upload failed", http.StatusInternalServerError)
		return
	}
	w.WriteHeader(http.StatusNoContent)
}

type URLResponse struct {
	URL string `json:"url"`
}

//encore:api public method=GET path=/avatars/:userID/url
func URL(ctx context.Context, userID string) (*URLResponse, error) {
	key := "users/" + userID
	ok, err := avatars.Exists(ctx, key)
	if err != nil {
		return nil, err
	}
	if !ok {
		return nil, &errs.Error{Code: errs.NotFound, Message: "no avatar"}
	}
	u, err := avatars.SignedDownloadURL(ctx, key, objects.WithTTL(15*time.Minute))
	if err != nil {
		return nil, err
	}
	return &URLResponse{URL: u.URL}, nil
}

encore run serves both endpoints against a local object store, so the upload flow works on a laptop without an AWS account. On deploy, Encore creates the S3 bucket and provisions an IAM role for the service, so no access keys sit in the repo or in CI. The bucket API, including signed upload URLs and public buckets, is in the Go object storage docs and the TypeScript object storage docs. An existing bucket can be attached with import an S3 bucket.

An AI coding agent can add a second bucket or a new upload endpoint in the same package and test it locally, and the bucket and its access arrive with the deploy.

Best fit: Go and TypeScript teams on AWS who want S3 buckets for each environment created from the code that uploads and downloads objects.

Good to know

• Bucket names are constrained by AWS naming rules and must stay unique in the namespace you use, so include the environment in the name when you create buckets by hand.
• Browser uploads need a CORS policy that allows PUT from your frontend's origin.
• Glacier classes that need a restore job before a read are a poor fit for anything the application serves on demand.

Related guides

• How to add object storage to a Go API
• Add an S3 bucket on AWS with an AI agent
• What is Google Cloud Storage?
• What is AWS IAM?
• How to deploy to AWS