Redis

Redis is an in-memory data store, commonly used as a cache, for counters and rate limits, for short-lived state like sessions and locks, and for lightweight queues.

Encore.rs lets you declare Redis databases in code. Encore provisions them for each environment: a managed Redis in the cloud, and a local one under encore run, with no setup. Under encore test, each test gets a database of its own.

Creating a database

Declare a database as a module-level static:

src/sessions.rs
use encore::storage::redis::Database;

static SESSIONS: Database = Database::new("sessions");

async fn record_visit(user: &str) -> Result<i64, encore::Error> {
    let key = format!("visits:{user}");
    let visits = SESSIONS.string().incr(&key, 1).await?;
    SESSIONS.generic().expire(&key, 24 * 60 * 60).await?;
    Ok(visits)
}

Encore reads the declaration from your source code to provision the database, so:

• It must be a module-level static, not a local variable or a value built inside a function.
• The name must be a string literal: lowercase letters, digits and dashes, starting with a letter.

The connection pool is built the first time the database is used, and reused after that.

Options

Options follow the name, each a method of its own:

use encore::storage::redis::{Database, EvictionPolicy};

// A cache: what it holds may be evicted, least recently used first.
static SESSIONS: Database = Database::new("sessions");

// Data that must survive a restart, and is never evicted.
static LEDGER: Database = Database::new("ledger").persistent();

// A cache that evicts the least frequently used keys first.
static THUMBNAILS: Database =
    Database::new("thumbnails").eviction_policy(EvictionPolicy::AllKeysLfu);

The policies are NoEviction, AllKeysLru, AllKeysLfu, AllKeysRandom, VolatileLru, VolatileLfu, VolatileTtl and VolatileRandom, with the same meaning as Redis's maxmemory-policy values. The Volatile policies only evict keys that have an expiry.

The options are part of the declaration: they take the database by value, so they can't be called on the static later.

Running commands

Commands are grouped by the data type they work on, each group reached through a method of the database:

Within a group, commands are named as in the redis crate and take what its conversion traits accept: integers, floats, bool, strings, byte slices, and Vecs, slices and tuples of them. A slice or a Vec of keys is how you pass several keys to commands like del and mget.

SESSIONS.string().set("name:1", "Ada").await?;
SESSIONS.string().set_ex("token:abc", "user-1", 3600).await?;
SESSIONS.hash().hset_multiple("user:1", &[("name", "Ada"), ("plan", "pro")]).await?;
SESSIONS.list().rpush("queue", &["job-1", "job-2"]).await?;
SESSIONS.sorted_set().zadd("scores", "ada", 42.0).await?;
SESSIONS.generic().del(&["name:1", "token:abc"]).await?;

Reading replies

Commands that reply with a fixed type, like exists (bool) or llen (u64), return it. Where a reply could be read several ways, as with get, the type is yours to choose:

use std::collections::HashMap;

let name: Option<String> = SESSIONS.string().get("name:1").await?;
let visits: u64 = SESSIONS.string().get("visits:ada").await?;
let user: HashMap<String, String> = SESSIONS.hash().hgetall("user:1").await?;
let top: Vec<(String, f64)> = SESSIONS.sorted_set().zrevrange_withscores("scores", 0, 9).await?;

Read a value that may be missing as an Option, which is None for a missing key. A reply that doesn't convert to the type you asked for fails with ErrCode::Internal.

incr takes an i64 or an f64, and replies with the same type, so incr("hits", 1) returns an i64 and incr("score", 0.5) an f64.

Expiry and conditional writes

Set an expiry with set_ex or generic().expire, or use set_options for SET's full set of options, re-exported from the redis crate:

use encore::storage::redis::{ExistenceCheck, SetExpiry, SetOptions};

// Take a lock for 30 seconds, unless someone holds it.
let reply: Option<String> = LOCKS
    .string()
    .set_options(
        format!("lock:{job}"),
        token,
        SetOptions::default()
            .conditional_set(ExistenceCheck::NX)
            .with_expiration(SetExpiry::EX(30)),
    )
    .await?;
let acquired = reply.is_some();

Structured values

Wrap a value in Json to store it as JSON, and read it back the same way. It works with any type that implements serde's Serialize and Deserialize:

use encore::storage::redis::Json;
use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize)]
struct Profile {
    name: String,
    plan: String,
}

SESSIONS.string().set("profile:1", Json(&profile)).await?;

let Json(profile): Json<Option<Profile>> = SESSIONS.string().get("profile:1").await?;

A missing value reads as JSON's null, so read one that may be missing as Json<Option<T>>.

To store a type of your own in another way, implement the ToRedisArgs and FromRedisValue traits from encore::storage::redis.

Call commands in one expression

Encore reads which commands each service runs on a database and grants the service the access it needs: reading or writing, per group. For that, a command has to be called in the same expression that names its group. A group that's stored or passed on says nothing about what it will be used for, and Encore reports it as an error:

// Good: Encore sees `get` on the string group.
let name: Option<String> = SESSIONS.string().get("name:1").await?;

// Error: Encore can't tell which command runs on `strings`.
let strings = SESSIONS.string();
let name: Option<String> = strings.get("name:1").await?;

Likewise, a Database can only be used by calling its methods on the static: it can't be stored, returned, or passed to a function. To hand a database to other code, take a handle.

Handles

A handle is a Copy value with the access its code needs, which you can pass around freely:

use encore::storage::redis::{Database, ReadOnlyDatabaseRef};

static SESSIONS: Database = Database::new("sessions");

// Library code takes a handle, and can only do what the handle allows.
async fn name_of(db: ReadOnlyDatabaseRef, user: &str) -> Result<Option<String>, encore::Error> {
    db.string().get(format!("name:{user}")).await
}

async fn greet(user: &str) -> Result<Option<String>, encore::Error> {
    name_of(SESSIONS.read(), user).await
}

The service is granted the handle's access to every group, whatever the code it's passed to does. The methods that make handles are const, so a handle can also initialize a static.

Pipelines

A pipeline sends several commands in one round trip, and returns their replies in order, as a tuple or a Vec. It's not a transaction: other clients' commands may run in between, and one command failing doesn't stop the rest.

let mut pipe = SESSIONS.pipeline();
pipe.string().get("name:1");
pipe.string().get("name:2");
let (first, second): (Option<String>, Option<String>) = pipe.exec().await?;

Queueing a command returns the queue, so commands can also be chained: pipe.string().get("a").get("b");.

Transactions

A transaction (MULTI/EXEC) runs its commands together, with no other client's in between. Queue the commands, then exec it:

let mut tx = LEDGER.transaction();
tx.string().set("account:ada", 100);
tx.hash().hset("owners", "ada", "Ada Lovelace");
tx.string().get("account:ada");
let ((), added, balance): ((), u64, i64) = tx.exec().await?;

Nothing is sent until exec, so a transaction is never left open across an .await. To abandon one, drop it, or call discard().

Watching keys

For a transaction that depends on values you read first, use watch. It runs your closure on a connection of its own: the closure reads what it needs through the connection and queues its changes on the transaction. The changes are applied only if none of the watched keys changed in between. If one did, the closure runs again.

let left: i64 = LEDGER
    .watch("account:ada", async |conn, tx| {
        let balance: i64 = conn.string().get("account:ada").await?;
        tx.string().set("account:ada", balance - 10);
        Ok(balance - 10)
    })
    .await?;

watch takes one key or several, and gives up with ErrCode::Aborted if the keys keep changing. The closure is an async closure, which needs Rust 1.85 or later.

For finer control, acquire() takes a Connection out of the pool, with its own watch, unwatch, transaction and pipeline. Its transaction's exec returns None if a watched key changed. The connection goes back to the pool when dropped.

Scripts

A Lua script runs on the server as one step. Declare it as a static with Script::new, then call it with its keys and arguments:

use encore::storage::redis::{Database, Script};

static LOCKS: Database = Database::new("locks");

static RELEASE: Script = Script::new(
    r#"
    if redis.call("get", KEYS[1]) == ARGV[1] then
        return redis.call("del", KEYS[1])
    end
    return 0
    "#,
);

async fn release(job: &str, token: &str) -> Result<bool, encore::Error> {
    LOCKS
        .script(&RELEASE)
        .key(format!("lock:{job}"))
        .arg(token)
        .invoke()
        .await
}

Each .key(…) is the next of the script's KEYS, and each .arg(…) the next of its ARGV. A script may only touch the keys it's passed through KEYS (see Keys).

A script declared with Script::read_only can only read, which the server enforces. It's the only kind a ReadOnlyDatabaseRef can run. To run a script inside a pipeline or a transaction, queue it: tx.script(&RELEASE).key(…).arg(…).queue();.

Scanning

The scan methods iterate over a database's keys, or over the elements of one key, a batch per round trip. Each returns a Scan, whose next returns the next element:

use encore::storage::redis::ScanOptions;

let mut keys = SESSIONS
    .generic()
    .scan::<String>(ScanOptions::default().pattern("user:*"));
while let Some(key) = keys.next().await {
    let key = key?;
    // …
}

ScanOptions::pattern keeps only what matches a glob-style pattern, and ScanOptions::count hints how many elements to look at per round trip. As with Redis's SCAN, an element present for the whole iteration is returned at least once, possibly more.

Publishing to channels

The pubsub() group is Redis's own channel messaging: a message reaches whoever is subscribed at that moment, and nobody else. For messages that are kept until they're handled, use Pub/Sub topics instead.

// Returns how many subscribers received the message.
SESSIONS.pubsub().publish("room:1", "hello").await?;

let mut room = SESSIONS.read().pubsub().subscribe("room:1").await?;
while let Some(message) = room.next().await {
    let text: String = message.payload()?;
    println!("{}: {text}", message.channel);
}

subscribe takes one channel or several, and psubscribe glob-style patterns. Each subscription has a connection of its own, which is closed when the subscription is dropped. A message's payload field holds the bytes as sent, and its payload::<T>() method converts them.

Keys

Every key is stored under a prefix of the database's own. That's what keeps databases sharing a server apart, and gives each test a database of its own. The prefix is added to every key a command takes and removed from every key a command returns, so your code never sees it. Two things follow:

• A script may only touch the keys it's passed through KEYS: a key built inside the script doesn't get the prefix.
• A client other than this SDK doesn't see the keys it writes under the same names. See below.

Using another Redis client

connection_string() returns a URL to connect to the database with another client:

let url = SESSIONS.connection_string()?;

That client connects past the database's key prefix: what it stores under a name is not what this SDK reads under that name. Calling connection_string grants the service read and write access to the database.

Using a database from another service

In a service other than the one that declares a database, bind it by name with Database::named:

use encore::storage::redis::Database;

static SESSIONS: Database = Database::named("sessions");

The name must be a string literal. A named database takes no options: it's configured where it's declared.

Errors

Commands return Result<_, encore::Error>. A command that fails, or a server that can't be reached, is ErrCode::Unavailable. A reply that doesn't convert to the type you asked for is ErrCode::Internal.

What's not supported

• Cache clusters and keyspaces, as in Encore.go and Encore.ts. Declare a database and choose your keys.
• Commands outside the groups above, including blocking commands such as BLPOP and server-wide commands such as FLUSHALL or KEYS. Use a script, a scan, or another client through connection_string.
• Transactions, pipelines and watch on a ReadOnlyDatabaseRef.

Testing

Under encore test, each test gets a Redis database of its own, empty unless a fixture fills it, so tests can run in parallel without seeing each other's keys. See Testing.