How to Access Cloud Storage From Cloud Run

A Cloud Run service that stores uploads, writes exports or serves files from Google Cloud Storage needs an identity and a role before its first request succeeds. Cloud Run runs every revision as a service account, and unless you choose one, that is the Compute Engine default service account, which the Cloud Run service identity docs say may be granted the Editor role on the project depending on your organization policy. The service then reaches every bucket in the project, along with most other resources, which is far more than one receipts bucket needs.

This guide sets up a receipts service that streams uploaded PDFs and images into a private bucket and back out to the user who uploaded them, first by hand with gcloud and the Go client library, then with Encore, where the bucket is declared in the service code and the service account comes with the deploy.

Cloud Storage access from Cloud Run at a glance

Create a service account for the service

Create a user-managed service account per Cloud Run service and deploy the service with it. The person or CI job that deploys needs the Service Account User role (roles/iam.serviceAccountUser) on that account, as the service identity configuration docs describe.

gcloud iam service-accounts create receipts-run \
  --display-name="receipts Cloud Run service"

gcloud storage buckets create gs://acme-receipts-prod \
  --location=europe-west1 \
  --uniform-bucket-level-access

# Object read and write on this one bucket, nothing at project level.
gcloud storage buckets add-iam-policy-binding gs://acme-receipts-prod \
  --member="serviceAccount:receipts-run@PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/storage.objectUser"

gcloud run deploy receipts \
  --image=europe-west1-docker.pkg.dev/PROJECT_ID/apps/receipts \
  --region=europe-west1 \
  --service-account=receipts-run@PROJECT_ID.iam.gserviceaccount.com \
  --set-env-vars=RECEIPTS_BUCKET=acme-receipts-prod

Bucket names are global, so acme-receipts-prod needs a suffix of your own.

Pick the narrowest bucket role

The Cloud Storage IAM roles reference lists three object roles that cover most services, and granting one on the bucket keeps the account away from every other bucket in the project:

New buckets should have uniform bucket-level access turned on, and with it enabled, ACLs are disabled and only bucket-level IAM grants access, so the binding above is the full list of who can read the receipts.

Use the client library without a key file

The Go client finds credentials through Application Default Credentials, which on Cloud Run is the attached service account returned by the metadata server. Leave GOOGLE_APPLICATION_CREDENTIALS unset in production, since a key file is a long-lived credential that keeps working after the service is gone. On a laptop, gcloud auth application-default login gives the same code your own user credentials.

package main

import (
	"context"
	"errors"
	"io"
	"log"
	"net/http"
	"os"

	"cloud.google.com/go/storage"
)

func main() {
	ctx := context.Background()

	// No key file: on Cloud Run the client gets tokens for the
	// service's identity from the metadata server.
	client, err := storage.NewClient(ctx)
	if err != nil {
		log.Fatal(err)
	}
	defer client.Close()
	bucket := client.Bucket(os.Getenv("RECEIPTS_BUCKET"))

	http.HandleFunc("PUT /receipts/{id}", func(w http.ResponseWriter, r *http.Request) {
		// Auth and id validation left out for brevity.
		obj := bucket.Object("receipts/" + r.PathValue("id"))
		ctx, cancel := context.WithCancel(r.Context())
		defer cancel()
		wr := obj.NewWriter(ctx)
		wr.ContentType = r.Header.Get("Content-Type")
		if _, err := io.Copy(wr, http.MaxBytesReader(w, r.Body, 10<<20)); err != nil {
			cancel() // abort the write; Close would finalize a truncated object
			http.Error(w, "upload failed", http.StatusBadRequest)
			return
		}
		// The object is only written once Close returns nil.
		if err := wr.Close(); err != nil {
			log.Printf("gcs write: %v", err)
			http.Error(w, "could not store receipt", http.StatusInternalServerError)
			return
		}
		w.WriteHeader(http.StatusCreated)
	})

	http.HandleFunc("GET /receipts/{id}", func(w http.ResponseWriter, r *http.Request) {
		rd, err := bucket.Object("receipts/" + r.PathValue("id")).NewReader(r.Context())
		if errors.Is(err, storage.ErrObjectNotExist) {
			http.NotFound(w, r)
			return
		} else if err != nil {
			log.Printf("gcs read: %v", err)
			http.Error(w, "could not read receipt", http.StatusInternalServerError)
			return
		}
		defer rd.Close()
		w.Header().Set("Content-Type", rd.Attrs.ContentType)
		io.Copy(w, rd)
	})

	port := os.Getenv("PORT") // set by Cloud Run
	if port == "" {
		port = "8080"
	}
	log.Fatal(http.ListenAndServe(":"+port, nil))
}

If io.Copy fails, including when MaxBytesReader rejects an oversized body, the code cancels the writer context so the partial upload is aborted rather than finalized by Close. The object is only written once Close returns nil. Auth, per-user prefixes and id validation are left out of this version to keep it short, and the Encore version below includes them.

Debugging 403 from Cloud Storage

1. Which account does the revision run as? Run gcloud run services describe receipts --region=europe-west1 and check the service account. If it is the Compute Engine default account, the code may only work because of Editor.
2. Does that account have a role on this bucket? Run gcloud storage buckets get-iam-policy gs://acme-receipts-prod and look for the account in a binding whose role covers the action.
3. Does the role cover the action? objectCreator cannot read and cannot overwrite an existing object, so a retry that writes the same name fails.
4. Is something above IAM blocking it? An organization policy or a VPC Service Controls perimeter around Cloud Storage denies requests that the bucket policy would allow.

Mounting the bucket as a volume instead

Cloud Run can also mount a bucket as a directory with Cloud Storage FUSE volume mounts, which suits code that expects files on disk, such as a model loaded at startup. The docs note that Cloud Storage FUSE is not a fully POSIX-compliant file system and provides no concurrency control for multiple writes to the same file, so the client library is the safer choice for uploads from many instances.

How does Encore give a Cloud Run service access to Cloud Storage?

Encore brings infrastructure from code to TypeScript and Go backends: a bucket declared in a service is part of that service's code, and when the app deploys to your own GCP account, Encore creates the Cloud Storage bucket and runs the service on Cloud Run, as the GCP infrastructure docs list. Each deployment gets its own service account with access to the buckets it uses, and the receipts code has no key file or bucket name in config.

Set up the app

Declare the bucket and the endpoints

// receipts/receipts.go
package receipts

import (
	"errors"
	"fmt"
	"io"
	"net/http"
	"regexp"

	"encore.dev"
	"encore.dev/beta/auth"
	"encore.dev/beta/errs"
	"encore.dev/storage/objects"
)

// A private bucket: local disk with `encore run`, Cloud Storage on GCP.
var Receipts = objects.NewBucket("receipts", objects.BucketConfig{})

const maxReceiptBytes = 10 << 20 // 10 MiB

var allowedTypes = map[string]bool{
	"application/pdf": true,
	"image/jpeg":      true,
	"image/png":       true,
}

var receiptID = regexp.MustCompile(`^[a-z0-9-]{1,64}$`)

// Each user's receipts live under their own prefix.
func keyFor(id string) (string, bool) {
	uid, _ := auth.UserID()
	if !receiptID.MatchString(id) {
		return "", false
	}
	return fmt.Sprintf("receipts/%s/%s", uid, id), true
}

// Put streams an uploaded receipt into the bucket.
//
//encore:api auth raw method=PUT path=/receipts/:id
func Put(w http.ResponseWriter, req *http.Request) {
	key, ok := keyFor(encore.CurrentRequest().PathParams.Get("id"))
	if !ok {
		http.Error(w, "invalid receipt id", http.StatusBadRequest)
		return
	}
	contentType := req.Header.Get("Content-Type")
	if !allowedTypes[contentType] {
		http.Error(w, "only pdf, jpeg and png", http.StatusUnsupportedMediaType)
		return
	}

	up := Receipts.Upload(req.Context(), key, objects.WithUploadAttrs(objects.UploadAttrs{
		ContentType: contentType,
	}))
	body := http.MaxBytesReader(w, req.Body, maxReceiptBytes)
	if _, err := io.Copy(up, body); err != nil {
		up.Abort(err) // nothing is stored for a cut-off or oversized body
		http.Error(w, "receipt too large or incomplete", http.StatusRequestEntityTooLarge)
		return
	}
	if err := up.Close(); err != nil {
		errs.HTTPError(w, err)
		return
	}
	w.WriteHeader(http.StatusCreated)
}

// Get streams a receipt back to the user who uploaded it.
//
//encore:api auth raw method=GET path=/receipts/:id
func Get(w http.ResponseWriter, req *http.Request) {
	key, ok := keyFor(encore.CurrentRequest().PathParams.Get("id"))
	if !ok {
		http.Error(w, "invalid receipt id", http.StatusBadRequest)
		return
	}
	attrs, err := Receipts.Attrs(req.Context(), key)
	if errors.Is(err, objects.ErrObjectNotFound) {
		http.Error(w, "receipt not found", http.StatusNotFound)
		return
	} else if err != nil {
		errs.HTTPError(w, err)
		return
	}

	r := Receipts.Download(req.Context(), key)
	if err := r.Err(); err != nil {
		errs.HTTPError(w, err)
		return
	}
	defer r.Close()
	w.Header().Set("Content-Type", attrs.ContentType)
	w.Header().Set("Content-Length", fmt.Sprint(attrs.Size))
	io.Copy(w, r)
}

Both endpoints use the auth access level, so they need an auth handler somewhere in the app that checks your sessions, and auth.UserID() puts each user's receipts under their own prefix. The object storage docs cover Upload, Download, Attrs and the other bucket operations, and the raw endpoint docs cover streaming request bodies.

encore run serves the bucket from local storage, so curl -X PUT -H "Content-Type: application/pdf" --data-binary @receipt.pdf localhost:4000/receipts/r-1 stores a receipt on a laptop with no GCP account, a GET by a different user returns 404, and a body over 10 MiB returns 413 with nothing stored.

Deploy to your GCP account

Connect your GCP project as described in Connect your cloud account, create an environment on GCP and run git push encore. Encore creates the bucket, the Cloud Run service and its service account, and the same Upload, Download and Attrs calls run against Cloud Storage. When a coding agent later adds a second bucket for exported reports, it declares it in the code, and the next deploy creates the bucket and grants the service access to it.

Best fit: TypeScript and Go teams on Cloud Run who want a service's bucket and its access to come from the service code.

Good to know

• Every Cloud Run revision runs as some service account, so a missing permission shows up as a 403 for a specific account and never as an empty credential chain.
• Roles granted at project level apply to every bucket in the project, including ones created later.
• A bucket in another project works the same way, with the binding added on that bucket for the Cloud Run service account.

How to choose

Write the gcloud or Terraform setup when the bucket is shared across teams, lives in another project or sits inside a VPC Service Controls perimeter you manage. Use Encore when the bucket belongs to one service and you want the bucket, its access and a working local version to come from the Go or TypeScript code that uses it.

Related guides

• What is Google Cloud Run?
• What is Google Cloud Storage?
• Upload files to Google Cloud Storage from Node.js
• How to connect Cloud Run to Cloud SQL
• Google Secret Manager on Cloud Run