How to Migrate Heroku Redis to Amazon ElastiCache

Heroku Key-Value Store, the add-on that used to be called Heroku Redis, usually ends up holding several unrelated things behind one REDIS_URL: page and API caches, rate limit counters, user sessions and the queue for a background worker. When the app moves to AWS, each of those needs a different answer, and copying the whole keyspace byte for byte is rarely the right one, since half of it would rebuild itself in minutes anyway.

This guide sorts the keys into what to drop, drain or copy, sets up Amazon ElastiCache in your own AWS account with Encore, and carries live sessions across with their remaining TTL so nobody is logged out at cutover. The hand-built ElastiCache setup is covered after it.

The migration at a glance

Step 1: Inventory the keyspace

Heroku attaches the add-on as REDIS_URL, a rediss:// URL, and all Key-Value Store plans require TLS with a self-signed certificate, as Heroku's own connection guide on Dev Center explains. With redis-cli --tls --insecure -u "$(heroku config:get REDIS_URL)", look at what is stored:

Group keys by prefix, note which prefixes have no TTL, and check the eviction policy with heroku redis:info. Keys without a TTL that your app treats as the only copy of some data are a migration problem of their own and belong in Postgres before anything else moves.

Step 2: Provision ElastiCache with Encore

Encore brings infrastructure from code to TypeScript and Go: a Redis cluster and its typed keyspaces are declared in the Go service that uses them, and the Encore platform provisions and deploys into your own AWS or GCP account. Deploying to AWS creates ElastiCache in the environment's private network, with the service connected through credentials Encore manages, as the AWS infrastructure docs describe.

Set up the app

The session keyspace replaces the session:<id> strings the Heroku app wrote by hand, and an endpoint guarded by a one-time token does the copy from Heroku, using the old URL stored as a secret:

// sessions/sessions.go
package sessions

import (
	"context"
	"crypto/subtle"
	"crypto/tls"
	"crypto/x509"
	"encoding/json"
	"errors"
	"time"

	"encore.dev/beta/errs"
	"encore.dev/rlog"
	"encore.dev/storage/cache"
	"github.com/redis/go-redis/v9"
)

// Dedicated session cluster: never evict a live session under memory pressure.
// Cache data belongs on a separate cluster with AllKeysLRU.
var cluster = cache.NewCluster("sessions", cache.ClusterConfig{
	EvictionPolicy: cache.NoEviction,
})

type Session struct {
	UserID    int64     `json:"user_id"`
	CreatedAt time.Time `json:"created_at"`
}

// Replaces the "session:<id>" string keys the Heroku app wrote by hand.
var Sessions = cache.NewStructKeyspace[string, Session](cluster, cache.KeyspaceConfig{
	KeyPattern:    "session/:key",
	DefaultExpiry: cache.ExpireIn(14 * 24 * time.Hour),
})

//encore:api private method=GET path=/sessions/:id
func Get(ctx context.Context, id string) (*Session, error) {
	s, err := Sessions.Get(ctx, id)
	if errors.Is(err, cache.Miss) {
		return nil, &errs.Error{Code: errs.NotFound, Message: "session expired"}
	}
	return &s, err
}

var secrets struct {
	HerokuRedisURL              string // the rediss:// URL from heroku config:get REDIS_URL
	HerokuRedisCA               string // PEM of the server CA, when you have one
	HerokuRedisAllowInsecureTLS string // set only to I_UNDERSTAND_UNVERIFIED_TLS for this backfill
	BackfillToken               string // a random value that authorizes the one-off copy
}

type BackfillParams struct {
	Token string `header:"X-Backfill-Token"`
}

type BackfillResult struct {
	Copied  int `json:"copied"`
	Skipped int `json:"skipped"`
}

// herokuTLSConfig verifies the Heroku Redis certificate when a CA PEM is
// provided. Heroku's own client examples disable verification because the
// service uses self-signed certificates and does not publish a CA; that
// fallback is opt-in for this one-off backfill only, never for ElastiCache.
func herokuTLSConfig(opts *redis.Options) (*tls.Config, error) {
	base := &tls.Config{MinVersion: tls.VersionTLS12}
	if opts.TLSConfig != nil && opts.TLSConfig.ServerName != "" {
		base.ServerName = opts.TLSConfig.ServerName
	}
	if secrets.HerokuRedisCA != "" {
		pool := x509.NewCertPool()
		if !pool.AppendCertsFromPEM([]byte(secrets.HerokuRedisCA)) {
			return nil, errors.New("HerokuRedisCA is not a valid PEM certificate")
		}
		base.RootCAs = pool
		return base, nil
	}
	if secrets.HerokuRedisAllowInsecureTLS == "I_UNDERSTAND_UNVERIFIED_TLS" {
		base.InsecureSkipVerify = true
		return base, nil
	}
	return nil, &errs.Error{Code: errs.FailedPrecondition, Message: "set HerokuRedisCA, or HerokuRedisAllowInsecureTLS=I_UNDERSTAND_UNVERIFIED_TLS for this backfill only"}
}

// BackfillSessions copies live sessions from Heroku Key-Value Store once,
// keeping each session's remaining TTL. Safe to re-run: it overwrites
// the same keys with the same values.
//
//encore:api public method=POST path=/admin/backfill-sessions
func BackfillSessions(ctx context.Context, p *BackfillParams) (*BackfillResult, error) {
	if secrets.BackfillToken == "" ||
		subtle.ConstantTimeCompare([]byte(p.Token), []byte(secrets.BackfillToken)) != 1 {
		return nil, &errs.Error{Code: errs.Unauthenticated, Message: "invalid token"}
	}
	opts, err := redis.ParseURL(secrets.HerokuRedisURL)
	if err != nil {
		return nil, err
	}
	tlsCfg, err := herokuTLSConfig(opts)
	if err != nil {
		return nil, err
	}
	opts.TLSConfig = tlsCfg
	src := redis.NewClient(opts)
	defer src.Close()

	res := &BackfillResult{}
	iter := src.Scan(ctx, 0, "session:*", 500).Iterator()
	for iter.Next(ctx) {
		key := iter.Val()
		raw, err := src.Get(ctx, key).Result()
		ttl, terr := src.PTTL(ctx, key).Result()
		if err != nil || terr != nil || ttl <= 0 {
			res.Skipped++ // expired or no TTL
			continue
		}
		var s Session
		if json.Unmarshal([]byte(raw), &s) != nil {
			res.Skipped++
			continue
		}
		id := key[len("session:"):]
		if err := Sessions.With(cache.ExpireIn(ttl)).Set(ctx, id, s); err != nil {
			return res, err
		}
		res.Copied++
	}
	rlog.Info("backfill done", "copied", res.Copied, "skipped", res.Skipped)
	return res, iter.Err()
}

The session cluster uses NoEviction, so Redis returns an error rather than deleting a live session when memory is full. Session keys still expire on their TTL. Cached data that is safe to evict belongs on a separate cluster with AllKeysLRU. The keyspace's KeyPattern is checked at compile time against every other keyspace on the cluster, which removes the prefix collisions that tend to build up in a long-lived Heroku Redis, and the struct type means the JSON encoding the old code did by hand is gone. See the caching docs for the other keyspace types. The backfill client verifies TLS when HerokuRedisCA is set; skipping verification is an explicit, one-off override because Heroku Key-Value Store uses self-signed certificates and does not publish a CA.

Set the secrets for production with encore secret set --type prod HerokuRedisURL, pasting the URL from heroku config:get REDIS_URL, encore secret set --type prod BackfillToken with a long random value, and encore secret set --type prod HerokuRedisCA with the PEM of the server CA. Heroku Key-Value Store uses self-signed certificates and does not publish a CA, so if you cannot obtain one, set HerokuRedisAllowInsecureTLS to I_UNDERSTAND_UNVERIFIED_TLS for this backfill only. The values are stored by Encore rather than in the repository, so neither a teammate's clone nor a coding agent in the codebase can read the Heroku credentials. Locally, encore run serves the keyspace from an in-memory Redis, so session code can be tested without a Heroku or AWS connection at all.

Best fit: teams moving a Heroku app to AWS who want ElastiCache in their own account with typed keys, and a session copy they can run and re-run from inside the new environment.

Provision ElastiCache by hand instead

The hand-built target is an ElastiCache replication group with a cache subnet group in private subnets, a security group that admits 6379 only from the app, in-transit encryption, and an auth token or RBAC user in Secrets Manager, as covered in the ElastiCache with Terraform guide. The client side has to switch from the Heroku URL to the new endpoint with TLS verification turned back on, since ElastiCache presents a certificate that standard clients accept. The copy script then has to run inside the VPC, on a task or an EC2 host, because ElastiCache has no public endpoint.

Step 3: Drain queues and switch writers

Workers that consume a Redis-backed queue should finish what is enqueued on Heroku before the cutover. Stop the processes that enqueue jobs, let the worker dynos empty the queues, then scale them to zero with heroku ps:scale worker=0. Jobs that fail during the drain land in the library's retry or dead set and are easier to deal with on Heroku than mid-copy. On AWS, a queue that needs delivery guarantees is a good moment to move to Pub/Sub with retries and a dead-letter queue, rather than recreating the Redis list.

Step 4: Copy sessions and cut over

Put the Heroku app in maintenance mode so no new sessions are created, then call the backfill once with curl -X POST -H "X-Backfill-Token: $TOKEN" https://<your-api>/admin/backfill-sessions, which runs inside the AWS environment and reaches ElastiCache over the private network. SCAN walks the keyspace in batches without blocking Heroku's Redis, and each session is written with the TTL it had left, so a session that would have expired in an hour still expires in an hour. Re-running it is harmless, since it writes the same values to the same keys.

Point DNS at the AWS deployment, check that a logged-in browser stays logged in, and watch the cache hit rate climb as entries are rebuilt. Once traffic is stable, delete the backfill endpoint and both secrets, and remove the add-on from Heroku. The Redis SCAN documentation covers the guarantees of the iteration, including that keys added during the scan may or may not be returned, which the maintenance window takes care of.

Related guides

• How to migrate from Heroku to AWS
• How to migrate Heroku Postgres to Amazon RDS
• How to replace Heroku Scheduler when you move to AWS
• Amazon ElastiCache with Terraform
• What is ElastiCache?