Nov 18, 20266 min read

S3 Multipart Upload: Large Files, Presigned Parts and Cleanup

When to use S3 multipart upload, the part and object limits after the 50 TB change, a presigned-part flow for browser uploads, cleaning up incomplete uploads, and streaming large files into a bucket declared in code.

Illustration for S3 Multipart Upload: Large Files, Presigned Parts and Cleanup

S3 multipart upload splits one object into parts that upload independently and are stitched together at the end. It is the only way past the 5 GB limit of a single PUT, and it is what makes large uploads survivable on a flaky connection, since a failed part is retried on its own.

An upload ID, part numbers, an ETag for every part, a final CompleteMultipartUpload call, and an abort path all have to be kept, or the parts sit in the bucket and are billed without ever appearing in a listing.

This guide covers the limits, a server-side and a browser-side flow, the CORS and cleanup details that trip people up, and how a bucket declared in application code handles multipart for you.

Multipart upload at a glance

Encore bucketMultipart via AWS SDKSingle PUT
Max object sizeSame as the provider50 TB5 GB
Who splits the fileThe runtime, once its upload buffer fillsYour code or @aws-sdk/lib-storageNobody
Retry granularityHandled by the runtimePer partWhole object
Abort on failureAutomatic in the runtimeYour code, plus a Lifecycle ruleNot needed
Browser direct uploadSigned upload URL, single PUTPresigned UploadPart URLsPresigned PutObject URL

Limits after the 50 TB change

Per the multipart limits table, parts are 5 MiB to 5 GiB, the last part can be smaller, and an upload can have 1 to 10,000 parts. In December 2025 AWS raised the maximum object size from 5 TB to 50 TB; reaching that requires multipart upload, and a single GET is still limited to 5 TB, so very large objects are read with concurrent ranged requests. The part count is the constraint to plan for: with 8 MiB parts, 10,000 parts caps an object at about 78 GiB, so pick the part size from the largest file you expect.

Server-side: let the SDK do it

For uploads that pass through your backend, @aws-sdk/lib-storage handles part splitting, concurrency and abort:

import { S3Client } from "@aws-sdk/client-s3"; import { Upload } from "@aws-sdk/lib-storage"; import { createReadStream } from "node:fs"; const s3 = new S3Client({}); export async function uploadExport(path: string, key: string) { const upload = new Upload({ client: s3, params: { Bucket: process.env.EXPORTS_BUCKET!, Key: key, Body: createReadStream(path) }, partSize: 16 * 1024 * 1024, // 16 MiB parts allow objects up to about 156 GiB queueSize: 4, // parts in flight leavePartsOnError: false, // abort the multipart upload if a part fails }); upload.on("httpUploadProgress", (p) => console.log(p.loaded, p.total)); await upload.done(); }

The task role needs s3:PutObject and s3:AbortMultipartUpload on the bucket prefix; CreateMultipartUpload, UploadPart and CompleteMultipartUpload are all authorized by s3:PutObject.

Browser-side: presigned parts

For files that should not pass through your servers, the backend starts the upload, presigns one URL per part, and completes it when the browser reports the ETags:

import { S3Client, CreateMultipartUploadCommand, UploadPartCommand, CompleteMultipartUploadCommand, } from "@aws-sdk/client-s3"; import { getSignedUrl } from "@aws-sdk/s3-request-presigner"; const s3 = new S3Client({}); const Bucket = process.env.UPLOADS_BUCKET!; export async function start(key: string, parts: number) { const { UploadId } = await s3.send(new CreateMultipartUploadCommand({ Bucket, Key: key })); const urls = await Promise.all( Array.from({ length: parts }, (_, i) => getSignedUrl(s3, new UploadPartCommand({ Bucket, Key: key, UploadId, PartNumber: i + 1 }), { expiresIn: 3600, }), ), ); return { uploadId: UploadId!, urls }; } export async function complete(key: string, uploadId: string, etags: string[]) { await s3.send(new CompleteMultipartUploadCommand({ Bucket, Key: key, UploadId: uploadId, MultipartUpload: { Parts: etags.map((ETag, i) => ({ ETag, PartNumber: i + 1 })) }, })); }

The two details that break this flow most often are CORS and ownership. The browser can only read each part's ETag response header if the bucket's CORS configuration lists ETag under ExposeHeaders, as described in the S3 CORS docs. And the backend should check that the key belongs to the caller before completing, because anyone holding the upload ID and the URLs can finish the object.

Clean up incomplete uploads

An upload that is started and never completed or aborted keeps its parts, and they are billed as storage. A Lifecycle rule with AbortIncompleteMultipartUpload is the safety net:

{ "Rules": [ { "ID": "abort-incomplete-uploads", "Status": "Enabled", "Filter": {}, "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 } } ] }

Apply it with aws s3api put-bucket-lifecycle-configuration, remembering that the call replaces the whole configuration. aws s3api list-multipart-uploads --bucket <name> shows what is pending today.

Large uploads into an Encore bucket

Encore

Encore brings infrastructure from code to the service that receives the file, and the bucket is declared next to the endpoint that writes to it. The object storage API gives you a writer; on S3 the runtime switches to a multipart upload once the data outgrows one upload buffer (10 MiB in the Go runtime, 8 MiB in the TypeScript runtime), uploads the parts, completes the upload on Close, and aborts it if the upload fails or you call Abort. encore run stores the objects on local disk, and on deploy to your own AWS account Encore provisions the S3 bucket and provisions an IAM role for the service.

Set up the app

Get started
brew install encoredev/tap/encore &&
encore app create
Copy

Run in your terminal to get started locally.

package exports import ( "io" "net/http" "encore.dev" "encore.dev/beta/errs" "encore.dev/storage/objects" ) var Exports = objects.NewBucket("exports", objects.BucketConfig{}) // Streams the request body into the bucket without holding it in memory. //encore:api auth raw method=PUT path=/exports/:name func UploadExport(w http.ResponseWriter, req *http.Request) { name := encore.CurrentRequest().PathParams.Get("name") writer := Exports.Upload(req.Context(), name) if _, err := io.Copy(writer, req.Body); err != nil { writer.Abort() // aborts the multipart upload on S3 errs.HTTPError(w, err) return } if err := writer.Close(); err != nil { // completes the multipart upload errs.HTTPError(w, err) return } w.WriteHeader(http.StatusCreated) }

The endpoint is a raw endpoint, so it streams the request body straight into the writer instead of buffering it, and the auth tag means the app also defines an auth handler, so only signed-in callers can write exports. For browser uploads that skip the backend, SignedUploadURL returns a URL for a single PUT, which covers files up to the 5 GB single-request limit; Encore does not expose presigned part URLs, so resumable multi-gigabyte browser uploads still need the hand-built flow above. An S3 lifecycle rule that aborts incomplete multipart uploads after a few days is still good practice on any bucket, including the one Encore provisions.

A coding agent can add an upload endpoint like this one and test it under encore run against local disk, then ship the same code to S3, where the runtime keeps track of the parts.

Best fit: Go or TypeScript backends that receive large exports or media through their API and want multipart handled by the runtime, with the bucket in their own AWS or GCP account.

Frequently asked questions

When should I use S3 multipart upload?

A single PUT is limited to 5 GB, so anything larger must use multipart upload, and AWS suggests it for large objects in general because parts upload in parallel and a failed part can be retried alone. Parts are 5 MiB to 5 GiB, except the last one, with at most 10,000 parts per upload, and objects can now be up to 50 TB.

Do incomplete multipart uploads cost money?

Yes. Parts of an upload that was started but never completed or aborted are stored and billed, and they do not show up as objects in the bucket listing. Add a Lifecycle rule with AbortIncompleteMultipartUpload, for example after 7 days, so abandoned uploads are cleaned up.

Deploy to your cloud on AWS and Google Cloud
without managing infrastructure

Encore lets developers and agents define application infrastructure directly in code, then automatically provisions it from local development to production in your AWS or GCP account.

~/orders — claude
Claude Code
Claude Code v2.1.180Opus 5.5~/orders