Tracing

Encore records distributed traces of what your app does. Encore.py traces the calls your code makes to Encore's infrastructure on its own. Requests to your web framework and your own operations are added through OpenTelemetry: you install Encore's tracer provider, instrument your framework with the standard OpenTelemetry package for it, and every span lands in the same trace.

What Encore traces automatically

Without any setup, Encore records a span for:

• SQL queries and transactions
• Redis commands
• Object storage operations
• Pub/Sub publishes
• The processing of each Pub/Sub message, as a trace of its own that links to the publish that sent it

Apart from message processing, each of these spans is a child of the OpenTelemetry span current in your code when you make the call. Once your framework is instrumented, a database query made while handling a request appears inside that request's trace.

Setting up OpenTelemetry

Encore records its own spans and metrics through its own providers, and doesn't install the OpenTelemetry globals, so it leaves any setup of your own alone. To send your app's spans, and those of instrumentation packages, to Encore, install Encore's providers as the globals when your app starts:

main.py
from opentelemetry import metrics, trace

from encore.metrics import meter_provider
from encore.tracing import tracer_provider

trace.set_tracer_provider(tracer_provider())
metrics.set_meter_provider(meter_provider())

Do this before you instrument your framework or create tracers. tracer_provider() and meter_provider() return the same provider every time you call them.

You can also pass the provider to a single instrumentation instead of installing it globally, for example FastAPIInstrumentor.instrument_app(app, tracer_provider=tracer_provider()).

Instrumenting your web framework

HTTP requests are traced by your framework's OpenTelemetry instrumentation package. For FastAPI, add opentelemetry-instrumentation-fastapi to your dependencies and instrument the app:

main.py
from contextlib import asynccontextmanager

import encore
from encore.metrics import meter_provider
from encore.tracing import tracer_provider
from fastapi import FastAPI
from opentelemetry import metrics, trace
from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor

trace.set_tracer_provider(tracer_provider())
metrics.set_meter_provider(meter_provider())


@asynccontextmanager
async def lifespan(app: FastAPI):
    await encore.start()
    yield


app = FastAPI(lifespan=lifespan)
FastAPIInstrumentor.instrument_app(app)

For Django, use opentelemetry-instrumentation-django:

from opentelemetry.instrumentation.django import DjangoInstrumentor

DjangoInstrumentor().instrument()

Other frameworks and HTTP clients have instrumentation packages of their own, such as opentelemetry-instrumentation-flask and opentelemetry-instrumentation-httpx. Instrumenting your HTTP client propagates the trace to the services you call.

Importing encore sets the OTEL_SEMCONV_STABILITY_OPT_IN environment variable to http, unless you've set it, so instrumentation packages record the stable HTTP semantic conventions that Encore expects. Instrumentation packages read the variable when they're imported, so import encore first.

Adding your own spans

Create spans with the standard OpenTelemetry API:

from opentelemetry import trace

from orders.db import db

tracer = trace.get_tracer(__name__)


async def get_user(user_id: int):
    with tracer.start_as_current_span("load user") as span:
        span.set_attribute("user.id", user_id)
        return await db.query("SELECT * FROM users WHERE id = $1", user_id).one()

The query's span is recorded as a child of load user, which in turn is a child of the request's span.

start_as_current_span also works as a decorator, including on async functions:

@tracer.start_as_current_span("recalculate prices")
async def recalculate_prices() -> None:
    ...

Spans support attributes, events, links and status as usual. An exception that leaves a start_as_current_span block is recorded on the span as an event, and sets the span's status to error. The current span follows your code across await and into tasks it starts with asyncio.create_task or asyncio.gather.

Encore decides which traces are sampled. A span in a trace that isn't sampled doesn't record anything, so its is_recording() returns False.

Logs in traces

With Encore's ConsoleHandler on your loggers, each log record carries the trace and span IDs of the span current when it's logged, which ties it to its place in the trace.

Viewing traces

Under encore run, traces are captured locally. View them in the local development dashboard, whose address encore run prints, or from the command line:

$ encore trace list              # the newest traces, most recent first
$ encore trace list --limit 10
$ encore trace get <trace-id>    # every event of one trace

For deployed environments, view traces in the Encore Cloud dashboard. See Distributed Tracing for more on what traces show.