
Wouldn't it be nice to store API keys, database passwords, and private keys directly in the source code?

Of course we can’t do that &ndash; it's horrifyingly insecure!
Unfortunately it's also [very common](https://www.ndss-symposium.org/ndss-paper/how-bad-can-it-git-characterizing-secret-leakage-in-public-github-repositories/).

So why does it happen? Because storing secrets securely used to be quite annoying.
Fortunately, Encore makes it easy.

## Defining secrets

With Encore you define secrets directly in your code:

```go
var secrets struct {
    SSHPrivateKey string    // ed25519 private key for SSH server
    GitHubAPIToken string   // personal access token for deployments
    // ...
}
```

<Callout type="important">

The variable must be an unexported struct named `secrets`, and all
the fields must be of type `string` like you see above.

</Callout>

Then, you can set the secret value using `encore secret set --<dev|prod> <name>`.
For example, `encore secret set --prod SSHPrivateKey`.

The values are stored safely using HashiCorp Vault, and delivered securely directly
to your production environment.

You can also set secret values for your development environments (including local development),
using `encore secret set --dev GitHubAPIToken`.

### Using secrets

Once you've provided values for all the secrets, you can just use them in your program
like a regular variable. For example:

```go
func callGitHub(ctx context.Context) {
    req, _ := http.NewRequestWithContext(ctx, "GET", "https:///api.github.com/user", nil)
    req.Header.Add("Authorization", "token " + secrets.GitHubAPIToken)
    resp, err := http.DefaultClient.Do(req)
    // ... handle err and resp
}
```

Secret keys are globally unique for your whole application; if multiple services use the same
secret name they both receive the same secret value at runtime.

Simply storing secrets securely

Store API keys and secrets


Encore is a fully-featured backend framework for rapidly building
serverless backends, perfect for combining with modern frontend applications.
It's designed for the modern cloud, and enables you to focus on building your product
without having to worry about infrastructure and unnecessary boilerplate.

Encore is also a platform, that lets you easily build, deploy, and manage your
cloud infrastructure, and supports deploying to all the major clouds.

## Designed for simplicity

Encore simplifies all the common backend development tasks, from writing code
to shipping to production, and even debugging when things go wrong.
It's designed from the ground up to provide an incredible developer experience.

With Encore you can focus on writing the business logic to shape your product.
With the business logic in place, Encore analyzes the code to understand how
your application works:

- What backend services you have defined
- The API functions each service exposes, and where they are called from other services
- What infrastructure resources (such as databases) each service requires

Encore uses this knowledge to automate everything else for maximum productivity,
such as:

- Setting up HTTP servers, encoding and decoding requests and responses
- Provisioning infrastructure for production and testing environments,
  with true serverless deployments
- Generating API documentation based on API request/response schemas
- Simplifying collaboration through Preview Environments for pull requests

Introduction


Building a modern, reliable backend application is a huge undertaking. Usually this involves:

- Multiple backend services, that each define their own API and call each other to orchestrate complex operations.

- Complex state management, often with several different databases, that is not only hard to manage
  but equally hard to understand.

- A hacked-together DevOps process consisting of many different tools, poorly tested scripts
  and a lack of consistency between different parts of the system that make improvements difficult.

By adopting Encore, these challenges become much more manageable. Encore is specifically designed
for building cloud backend applications, and to make these use cases simpler. Creating a new service
with Encore is as easy as creating a new Go package, and calling another API is as easy as a function call.

As you define new services and databases, Encore analyzes your application code to build up a model
of exactly how it works and what infrastructure each component requires. When it comes to deploying
your application, Encore handles your whole DevOps pipeline and makes sure everything is consistently
provisioned and configured.

## A state of the art workflow

Encore is all about providing the best development workflow you can imagine.
Traditional software development processes are disjointed, consisting of many disparate tools
that barely fit together into a whole. Often accomplishing even the simplest tasks involves
lots of manual steps and glue work.

Encore reimagines how backend development should be like. A single command to run your whole
application. Access to all your infrastructure at your fingertips, regardless of environment.
A much faster feedback loop through features like Hot Reload, tracing for local development,
automatic and always up-to-date API documentation and generated, type-safe API clients.

## From prototype to production

Many frameworks and tools designed for rapid development are great for prototyping, but become
a liability as your application grows. *Encore is different.* It's designed from the ground up to
be suitable for large scale software engineering. On top of this foundation are various features
designed to make it incredibly easy to get started. In this way you get the benefits of incredibly
rapid prototyping, while your application gracefully handles increased requirements and scale
as you move to production and beyond.

## Collaborate without silos

Building modern applications is done in teams. At the same time our development tools haven't caught up.
We're still working on code in silos. Sharing our work happens only when everything is done, usually
in the form of a pull request.

Encore makes it easy to co-develop, working interactively on the frontend and backend.
Through static analysis it builds up a detailed Application Model that helps
non-technical people make sense of your application, and enables features like a built-in
CMS that works directly on top of your data model.

Encore helps you build better software, faster

Why Encore?


Setting up Encore is easy. Simply open your terminal and run the relevant install command for your system.

Mac OS:
```bash
brew install encoredev/tap/encore
```

Windows:
```bash
iwr https://encore.dev/install.ps1 | iex
```

Linux:
```bash
curl -L https://encore.dev/install.sh | bash
```

## Install Docker

To set up SQL databases for local development, Encore uses Docker.
To run those applications, [install Docker](https://docs.docker.com/install/).

After installing it, check that it works by running `docker version` from your shell.
If it works, you should see something like this:

    Docker version 18.09.2, build 6247962

## Next steps

Now that you have Encore up and running, get started by [creating your first Encore app](/docs/tutorial/create-app).

Installation

## Install the Encore CLI
To work with Encore you need our command-line utility. Install by running the appropriate command for your system.

Mac OS:
```bash
brew install encoredev/tap/encore
```

Windows:
```bash
iwr https://encore.dev/install.ps1 | iex
```

Linux:
```bash
curl -L https://encore.dev/install.sh | bash
```
## Make sure Docker is installed and running
To create Encore applications with databases, you need Docker installed and running. If you haven't already, [install Docker](https://docs.docker.com/install/) before proceeding.

## Create your app
Create your app by running:
```bash
encore app create
```

Then press `Enter` to create your free account, following instructions on screen.

Coming back to the terminal, pick a name for your app.

Then select the `Hello World` app template.

## Run your app

Open the folder created for your application. (Note: Your App ID will be different)
```bash
cd hello-world-4x3b
```

Then while in the app root directory, run your app:
```bash
encore run
```

You should see this:

```bash
$ encore run
Running on http://localhost:4060
9:00AM INF registered endpoint endpoint=World service=hello
```

While you keep the app running, open a separate terminal and call your API endpoint:

```bash
$ curl http://localhost:4060/hello.World
{"Message": "Hello, world!"}
```

If you see this message, you've successfully created and run your first Encore application.
Well done!

## Open the Encore web application
While running your app, you can monitor local logs and traces by opening [http://localhost:4060](http://localhost:4060) in your browser.

Now head to [the Encore web application](https://app.encore.dev) where you can see production logs and traces, manage environments and configure your cloud hosting.


Get started with Encore quickly

Speedrun


This tutorial will introduce you to Encore and the complete cycle of building
a production backend application, including setting up an app, development, deployment,
and calling it from a web frontend. It assumes you are comfortable with the command line,
Git, and some experience writing Go.

If you haven't already, [install Encore](/docs/intro/install) before continuing
with this tutorial.

Then get started with the first step: [creating your first Encore app](/docs/tutorial/create-app)!

Tutorial


In this tutorial you will create your very first Encore application,
and deploy it to production using the Encore platform.

## 1.1 Make sure Encore is installed
If you haven't already done so, [install Encore](/docs/intro/install).
If you're not sure, run `encore version` in your terminal.
It should print something like:

```
encore version v1.0.0
```
## 1.2 Make sure Docker is installed and running

To create Encore applications with databases, you need Docker installed and running.
If you haven't already, [install Docker](https://docs.docker.com/install/) before proceeding.


## 1.3 Create a new app
Once you've installed Encore, creating an app is easy, simply run:
```bash
encore app create
```

Then pick a name for your app using lowercase letters, digits, or dashes.

Then choose between two example applications `Hello World` and `Trello Clone`, or creating an `Empty app`.

Once created, take a note of its **App ID**. This will be something like `hello-world-4x3b` (yours will be different).

## 1.4 Running your app

To run your app, open the folder created for your application. (Note: Your App ID will be different)
```bash
cd hello-world-4x3b
```

Then while in the app root directory, run your app by simply running:
```bash
encore run
```

You should see this:

```bash
$ encore run
Running on http://localhost:4060
9:00AM INF registered endpoint endpoint=World service=hello
```

That means your application is up and running!

While you keep the app running, open a separate terminal and call your API endpoint:

```bash
$ curl http://localhost:4060/hello.World
{"Message": "Hello, world!"}
```

If you see this message, you've successfully created and run your first Encore application.
Well done!

## 1.5 Open the Encore web application
While running your app, you can monitor local logs and traces by opening [http://localhost:4060](http://localhost:4060) in your browser.

Now head to [the Encore web application](https://app.encore.dev) where you can see production logs and traces, manage environments and configure your cloud hosting.

## 1.6 Your first API
<Callout type="information">
If you started with the `Empty app`, here are instructions for how you create your first API.
If you picked `Hello World`, this will already be set up in your app.
</Callout>

For an app to be useful, it must have an API. Each API belongs to a *service*.
Start by creating a new directory named `hello` in your app root directory.
This which will be your service package.
Then, create a file named `hello.go` inside:

```bash
$ cd myapp
$ mkdir hello
$ touch hello/hello.go
```

Open up the `hello.go` file and add the following contents:

```go
package hello

import "context"

type Response struct {
    Message string
}

//encore:api public
func World(ctx context.Context) (*Response, error) {
    return &Response{Message: "Hello, world!"}, nil
}
```


1. Create a new Encore app


In step 1 you created your first Encore app and ran it locally.
Now is the time to deploy it to production, using the Encore Platform.

Encore automatically creates a production environment named `prod`
and deploys to it automatically when you push.

## Commit and push your change

Now it's time to push our freshly coded API and see it deployed!

Go back to your terminal, and commit the `hello` service:

```bash
$ git add hello
$ git commit -m 'Add hello service'
$ git push
```

Git will output a bunch of stuff, but towards the end you should see something like this:
```bash
remote: refs/heads/main: triggered build yhgznbufao
To encore://hello-world-4x3b
   84a7586..17f9750  main -> main
```

That means a build has been triggered, and if that succeeds Encore will deploy your change.

Head over to `app.encore.dev` again, open your app, and open the `Builds` page and you should
see the build you just triggered.

Once it completes successfully, click the `Deploy` menu item under the `prod` environment in the menu on the left.
Here you will see the deploy once the has build completed successfully.

Once the deploy completes, your app is up and running in production!

### Call your API
To verify that it's running, let's call our API. There are two ways to do this:
through the Encore Platform, and directly from the terminal. Let's do both!

On `app.encore.dev` with your app open, click the `API` menu item in the menu on the left.
You should see the API Documentation for your app, with the `hello` service and its `World` endpoint.

Next to the `World` endpoint, click the `Call` button on the right-hand side.
Then, click `Send` to make an API call.
You should see the response: `{"Message": "Hello, World!"}`.

Now, open your terminal and run (replace `hello-world-4x3b` with your own App ID):

```bash
$ curl https://hello-world-4x3b.encoreapi.com/prod/hello.World
{"Message": "Hello, world!"}
```

If you see this, you've successfully deployed and made an API call to your very first Encore app in production.
Nicely done!

Move on to the next step of the tutorial to add a SQL database to your service.

2. Deploy your app to the cloud


In the previous step you deployed your app to production using the Encore Platform.
Now you'll add a SQL database to store some data.

## Add a new API endpoint

We'll add a new API endpoint, `hello.There` that takes a name as input
and responds with a personalized greeting. The database will store
the names of people and how many times you've met.

Create a new file named `hello/there.go` with the following contents:

```go
package hello

import (
  "context"
  "fmt"
)

type ThereParams struct {
  Name string
}

// There responds with a personalized greeting.
//encore:api public
func There(ctx context.Context, params *ThereParams) (*Response, error) {
  greeting, err := generateGreeting(params.Name)
  if err != nil {
    return nil, err
  }
  return &Response{Message: greeting}, nil
}

// generateGreeting generates a personalized greeting for name.
func generateGreeting(ctx context.Context, name string) (string, error) {
  msg := fmt.Sprintf("Hello, %s!", params.Name)
  return msg, nil
}
```

Run your app with `encore run`.
Open Encore's developer dashboard by visiting [http://localhost:4060](http://localhost:4060) in your browser.

Navigate to the API Documentation by clicking **API** in the top left.
There you'll see your new `hello.There` endpoint.
Make an API Call by clicking on `Call` and enter `"Jane"` as your name.

You'll immediately see a personalized response:
```json
{"Message": "Hello, Jane!"}
```

It works! Now it's time to add a database to track how many times you've met.

## Create a database table

Inside your `hello` service, add a new directory named `migrations`.

Inside this folder, create a new file named `1_create_table.up.sql`, with the
following contents:

```sql
CREATE TABLE people (
  name TEXT PRIMARY KEY,
  count INT NOT NULL
);
```

This will create a database table named `people`, with `name` as the primary key
along with `count` tracking how many times you've met.

## Query the database

Now it's time to use the database in your API.
Go back to `hello/there.go` and modify the `import` declaration at the top
to include `encore.dev/storage/sqldb`:

```go
import (
  "context"
  "fmt"

  "encore.dev/storage/sqldb"
)
```

Then, modify the `generateGreeting` function at the bottom to look like this:

```go
func generateGreeting(ctx context.Context, name string) (string, error) {
  // Make an "UPSERT" (insert or update) query,
  // inserting a row if it doesn't already exist
  // and otherwise updating it by incrementing count.
  var count int
  err := sqldb.QueryRow(ctx, `
    INSERT INTO "people" (name, count)
    VALUES ($1, 1)
    ON CONFLICT (name) DO UPDATE
    SET count = people.count + 1
    RETURNING count
  `, name).Scan(&count)
  if err != nil {
    return "", err
  }
  
  var greeting string
  if count > 1 {
    greeting = fmt.Sprintf("Hey again %s! We've met %d time(s) before.", name, count-1)
  } else {
    greeting = fmt.Sprintf("Nice to meet you, %s!", name)
  }
  return greeting, nil
}
```

This will query the database, and insert a row for the given name with `count = 1`.
If the row already exists, it instead increments `count`.
Finally it returns the stored `count` value.

We scan this value into the `count` variable and then generate a personalized
greeting based on the count.

Try it out by running `encore run`. Either open the API Documentation and try again,
or run in a separate terminal:

```bash
$ curl http://localhost:4060/hello.There -d '{"Name": "John"}'
{"Message": "Nice to meet you, John!"}
$ curl http://localhost:4060/hello.There -d '{"Name": "John"}'
{"Message": "Hey again John! We've met 1 time(s) before."}
$ curl http://localhost:4060/hello.There -d '{"Name": "John"}'
{"Message": "Hey again John! We've met 2 time(s) before."}
```

Wonderful, it works!

## Deploy to production

To deploy this to production, simply push this code to Encore:

```bash
$ git add -A
$ git commit -m "Add hello.There endpoint with a personalized greeting"
$ git push
[... stuff ...]
remote: refs/heads/main: triggered build nba83nfoa2
To encore://hello-world-4x3b
   84a7586..17f9750  main -> main
```

Head over to your app on [app.encore.dev](https://app.encore.dev).
Once the deploy completes, go to the API page and make an API call
to your new `hello.There` endpoint.

You should see the same thing as before:
```json
{"Message": "Nice to meet you, John!"}
{"Message": "Hey again John! We've met 1 time(s) before."}
```

Excellent! You now have a database up and running in production,
and an API hooked up to it.

## Next steps
This concludes the Encore tutorial, but there's lots more you can do with Encore.
Explore the documentation, notably the [How-to Guides](/docs/how-to) to get more
ideas on how you can use Encore to build better software faster.

3. Add a SQL database


Welcome! Encore is a platform for rapidly building backend applications.
Use the Encore&nbsp;Framework to rapidly develop backend applications, and then use
Encore&nbsp;Server to collaborate with your team and ship with confidence.

- [The Encore Platform](/docs/concepts/platform)
- [Encore app structure](/docs/concepts/app-structure)
- [Defining services and APIs](/docs/concepts/services-and-apis)
- [Using SQL Databases](/docs/concepts/databases)
- [Scaling your app](/docs/concepts/scaling)
- [Automated Testing](/docs/concepts/testing)
- [Authentication](/docs/concepts/auth)
- [Environments](/docs/concepts/environments)
- [Automated API Docs](/docs/concepts/api-docs)
- [Security](/docs/concepts/security)

Understand the Encore Platform

Concepts


The Encore platform helps you develop, deploy, and debug backend APIs
faster than ever before. With Encore you can focus on what makes your application
unique, instead of spending time on "boilerplate" &mdash; the boring, repetitive work
you traditionally spend a lot of time on.

![The Encore Platform](/assets/img/encore-platform.png)

## 1. Develop backends with the Encore Framework

Setting up a productive development environment for building a modern backend application
is very time-consuming. Many different services that need to be integrated; complex
configuration to connect all the pieces together; and many manual steps.

The Encore framework lets you instead get back to what matters &ndash; building your application.
The Encore framework is a powerful way to write backend APIs using Go, that takes away the pain
of building production-ready backend applications:

- It simplifies managing backend services, creating APIs, calling other APIs, and so on.
  It generates API documentation and type-safe API clients for your frontend out of the box.

- It sets up and migrates your databases, handles connections and database passwords securely.

- It simplifies your workflow, getting you up and running in seconds.
  Setting everything up is as easy as `encore run`.

## 2. Collaborate effortlessly

As software quickly takes over the world, the need for a collaborative development process
becomes even greater. Encore Server provides a suite of collaboration tools that integrate
effortlessly with applications built with Encore, including:

- Each Pull Request automatically gets a dedicated Preview Environment,
  where you can easily verify the change &ndash; and test your frontend, too!

- Encore generates API Documentation for your application through its static analysis.
  It also generates type-safe API Clients for your frontend!

- Co-develop a frontend and backend in realtime by exposing your local machine
  as a bespoke environment using `encore run --tunnel`. Perfect for remote work.

## 3. Ship with confidence and speed

Instead of spending time setting up complicated build and deployment pipelines,
provisioning Kubernetes clusters, databases, and more, Encore Server takes care
of setting it all up for you.

You can either use Encore Cloud and deploy in true serverless fashion, or connect
with your own cloud account and enable Encore to deploy straight there.

Encore applications automatically scale horizontally, can be run in one or multiple
regions, and even across multiple cloud providers.

### Debug production issues

Encore comes with state of the art Tracing functionality, which dramatically
simplifies finding the source of production issues. Is a service slow to respond?
What data was being passed in to cause a particular bug to surface? Tracing makes
it easy to find the answer.

Build better backends, faster

The Encore Platform


Encore is designed around the **application**.
A single Encore application should contain all the backend logic for your whole product.

Don't worry about splitting up a large application into multiple smaller ones;
Encore is designed to scale to large code bases, and provides the best developer experience
when all the functionality is contained in the same Encore application.

## Structure

Encore divides applications into systems, services, and components.
*See the [backend architecture best practices](/docs/engineering/backend-architecture) guide
for more information on this structure.*

In Encore, services are represented as **Go packages** (see [defining services](/docs/oncepts/services-and-apis) for more information).
To group related services together into a system, you can move the related service packages within
a subdirectory (that represents the system) within the project.

### Example

For example, a Trello clone might consist of two systems: the **Trello** system (for managing trello boards & cards),
the **User** system (for user and organization management, and authentication), and the **Premium** system (for subscriptions
and paid features).

On disk it might look like this:

```go
/my-trello-clone
├── encore.app       // ... and other top-level project files
│
├── premium          // premium system
│   ├── payment      // payment service
│   └── subscription // subscription service
│
├── trello           // trello system
│   ├── board        // board service
│   └── card         // card service
│
└── usr              // user system
    ├── org          // org service
    └── user         // user service
```

Each service consists of Go files that implement the service business logic,
database migrations for defining the structure of the database(s), and so on.
See the docs on [defining services](/docs/concepts/services-and-apis) for more information
on what this structure looks like.

Systems, services, APIs, oh my

App structure


Encore divides applications into systems, services, and components.
*See the [backend architecture best practices](/docs/engineering/backend-architecture) guide
for more information on this structure.*

## Defining a service

An Encore service consists of a regular Go package that defines **one or more APIs**.
The package name is the service name.

Within a service, you can have multiple packages; this is a good way to define components.
Note that only the **service package** can define APIs; sub-packages within a service
cannot themselves define APIs. However, if it makes sense you can define an API in the service package
that simply calls a function within one of the sub-packages.

## Defining APIs

So if what defines a service is having one or more APIs, how do we define an API, then? It's simple:

```go
package hello // service name

// PingParams is the request data for the Ping endpoint.
type PingParams struct {
    Name string
}

// PingResponse is the response data for the Ping endpoint.
type PingResponse struct {
    Message string
}

// Ping is an API endpoint that responds with a simple response.
// This is exposed as "hello.Ping".
//encore:api public
func Ping(ctx context.Context, params *PingParams) (*PingResponse, error) {
    msg := fmt.Sprintf("Hello, %s!", params.Name)
    return &PingResponse{Message: msg}, nil
}
```

In other words, you define a regular function and use the `//encore:api` annotation
to tell Encore that this is an API.

### Access controls

In fact, when you define an API you have three options in terms of how the API can be accessed:

* `//encore:api public` &ndash; defines a public API that anybody on the internet can call
* `//encore:api private` &ndash; defines a private API that only other backend services can call
* `//encore:api auth` &ndash; defines a public API that anybody can call, but that requires valid authentication.

For defining APIs that require authentication, see the [authentication guide](/docs/concepts/auth).

This approach is simple, but really powerful. It lets Encore use [static analysis](/docs/concepts/application-model)
to understand the request and response schemas of all your APIs, which enables it to automatically generate API documentation
and type-safe API clients, and much more.

### Request and response schemas

APIs in Encore consist of a regular function with a request data type and a response data type.
In the example above we have both: the request data is of type `PingRequest` and the response data of type
`PingParams`. This is usually the case, but in fact they're both optional in case you don't need them.
That means there are four different ways of defining an API:

* `func Foo(ctx context.Context, p *Params) (*Response, error)` &ndash; when you need both
* `func Foo(ctx context.Context) (*Response, error)` &ndash; when you only return a response
* `func Foo(ctx context.Context, p *Params) error` &ndash; when you only respond with success/fail
* `func Foo(ctx context.Context) error` &ndash; when you need neither request nor response data

As you can see there are two parts always present: the `ctx context.Context` parameter and the `error` return value.

The `ctx` parameter is used for *cancellation*. It lets you detect when the caller is no longer interested in the result,
and therefore lets you abort the request processing and save resources that nobody needs.
[Learn more about contexts on the Go blog](https://blog.golang.org/context).

The `error` return value is always important because from the caller's perspective, API calls **always** can fail.
Therefore even though our simple `Ping` API endpoint above never fails in its implementation, from the perspective
of the caller perhaps the service is crashing or the network is down and the service cannot be reached.

Why does this matter to the implementation, you ask? Because in Encore, *API calls look like function calls*.

## Calling an API
Calling  an API endpoint with Encore looks like a regular function call. Import the service package as if it's a regular
Go package, and then call the API endpoint as if it's a regular function.

For example:
```go
import "app.encore.dev/myapp/hello" // import service

//encore:api public
func MyOtherAPI(ctx context.Context) error {
    resp, err := hello.Ping(ctx, &hello.PingParams{Name: "World"})
    if err == nil {
        log.Println(resp.Message) // "Hello, World!"
    }
    return err
}
```

When building your application, Encore uses [static analysis](/docs/concepts/application-model) to find all
API calls and compiles them to proper API calls. This provides all the benefits of function calls, like 
compile-time checking of all the parameters and auto-completion in your editor,
while still allowing the division of code into logical components, services, and systems.

## Defining databases

Each service defines the infrastructure resources it uses simply by using them in code.
To learn more about how this works, see the guide on using [SQL databases](/docs/concepts/databases).

Simplifying (micro-)service development

Defining APIs and services


Encore treats SQL databases as logical resources.
This means using a database only requires you to [define the schema](#defining-a-database-schema)
and then start using it. Encore takes care of provisioning the database, running
new schema migrations during deploys, and connecting to it.

Encore's SQL databases are **PostgreSQL** databases.

## Defining a database schema

Database schemas are defined by creating *migration files* in a directory named `migrations`
within an Encore service package. Each migration file is named `<number>_<name>.up.sql`, where
`<number>` is a sequence number for ordering the migrations and `<name>` is a
descriptive name of what the migration does.

Each migration runs in order, and expresses the change in the database schema
from the previous migration.

The first migration usually defines the initial table structure. For example,
a `todo` service might start out by creating `todo/migrations/1_create_table.up.sql` with
the following contents:

```sql
CREATE TABLE todo_item (
    id BIGSERIAL PRIMARY KEY,
    title TEXT NOT NULL,
    done BOOLEAN NOT NULL DEFAULT false
);
```

## Querying databases

Once you have defined a database schema, you can easily query it.
Simply import `encore.dev/storage/sqldb` in your service package (or any sub-packages within the service),
and start using the package functions. The interface is similar to that of the Go standard library's
`database/sql` package.

For example, to read a single todo item using the example schema above:

```go
var item struct {
    ID int64
    Title string
    Done bool
}
err := sqldb.QueryRow(ctx, `
    SELECT id, title, done
    FROM todo_item
    LIMIT 1
`).Scan(&item.ID, &item.Title, &item.Done)
```

### Connecting to databases

It's often useful to be able to connect to the database from outside the backend application.
For example for scripts, ad-hoc querying or dumping data for analysis.

The Encore CLI comes with built in support for this:

* Use `encore db shell [--env=<name>] <service-name>` to open a [psql](https://www.postgresql.org/docs/current/app-psql.html)
  shell to the database for `<service-name>` in the given environment.
  Leaving out `--env` defaults to the local development environment.

* Use `encore db proxy [--env=<name>]` to create a local proxy that forwards any incoming connection
  to the database in the given environment.
  Leaving out `--env` defaults to the local development environment.

See `encore help db` for more information on database management commands.

## Provisioning databases

Encore automatically provisions databases in a suitable way depending on the environment.
When running locally, Encore creates a database cluster using docker.
In the cloud, how the database is provisioned depends on the type of [Encore Environment](/docs/concepts/environments):

- In `production` environments, the database is provisioned through the Managed SQL Database
  service offered by the chosen cloud provider.
- In `development` environments, the database is provisioned as a Kubernetes deployment
  with a persistent disk attached.

Encore automatically provisions databases to match what your application requires.
Simply define a database schema ([see above](#defining-a-database-schema)) and Encore
will provision the database at the start of the next deploy.


Provisioning, migrating, querying

Using SQL databases


Encore applications automatically scale up and down to meet your traffic demands.
Whether that means a single user or hundreds of millions, building your application
with Encore gives you the peace of mind that your application scales to meet your needs
while at the same time reducing your cloud bill.

Under the hood this is implemented using Kubernetes autoscaling groups,
which monitor the CPU and memory usage of your application and determines
how many instances should be started or shut down to match your traffic requirements.

Whether you use Encore Cloud or deploy to your own cloud, all of this is handled
for you so you don't have to think about it. All so you can focus on what matters:
building your product.

From zero to one to millions

Scaling your application


Go comes with excellent built-in support for automated tests.
Encore builds on top of this foundation, and lets you write tests in exactly the same way.
We won't cover how to write tests here; see [the official Go docs](https://golang.org/pkg/testing/) for that.

The main difference is that due to Encore requiring an extra compilation step,
you must run your tests using `encore test` instead of `go test`. This is
a wrapper that compiles the Encore app and then runs `go test`, and supports
all the same flags that the `go test` command does.

For example, use `encore test ./...` to run tests in all sub-directories,
or just `encore test` for the current directory.

## Integration testing

Since Encore removes a lot of boilerplate, a lot of the code you'll be writing
is business logic that involves databases and calling APIs between services.
Such behavior is most easily tested with integration tests.

When running tests, Encore automatically sets up the databases you need
in a separate database cluster. They are additionally configured to skip `fsync`
and to use an in-memory filesystem since durability is not a concern for automated tests.

This drastically reduces the speed overhead of writing integration tests.

In general, Encore applications tend to focus more on integration tests
compared to traditional applications that are heavier on unit tests.
This is nothing to worry about and is the recommended best practice.

Confidence at speed

Automated testing


Almost every application needs to know who's calling it, whether the user
represents a person in a consumer-facing app or an organization in a B2B app.
Encore supports both use cases in a simple yet powerful way.

As described in [defining APIs](/docs/concepts/services-and-apis), Encore offers three access levels
for APIs:

* `//encore:api public` &ndash; defines a public API that anybody on the internet can call
* `//encore:api private` &ndash; defines a private API that only other backend services can call
* `//encore:api auth` &ndash; defines a public API that anybody can call, but that requires valid authentication.

When an API is defined with access level `auth`, outside calls to that API must specify
an authorization header, in the form `Authorization: Bearer <token>`. The token is passed to
a designated auth handler function and the API call is allowed to go through only if the
auth handler determines the token is valid.

## The auth handler

Encore applications can designate a special function to handle authentication,
by defining a function and annotating it with `//encore:authhandler`. This annotation
tells Encore to run the function whenever an incoming API call contains an authentication token.

The auth handler is responsible for validating the incoming authentication token
and returning an `auth.UID` (a string type representing a **user id**). The `auth.UID`
can be whatever you wish, but in practice it usually maps directly to the primary key
stored in a user table (either defined in the Encore service or in an external service like Firebase or Okta).

<Callout type="important">

If your auth handler returns an error, Encore responds with a `500 Internal Server Error`.
To signify that the token is not valid, return the empty string (`""`) as the `auth.UID`.
Encore will then respond with `401 Unauthorized.

</Callout>

### With custom user data

Oftentimes it's convenient for the rest of your application to easily be able to look up
information about the authenticated user making the request. If that's the case,
define the auth handler like so:

```go
import "encore.dev/beta/auth"

// Data can be named whatever you prefer (but must be exported).
type Data struct {
    Username string
    // ...
}

// AuthHandler can be named whatever you prefer (but must be exported).
//encore:authhandler
func AuthHandler(ctx context.Context, token string) (auth.UID, *Data, error) {
    // Validate the token and look up the user id and user data,
    // for example by calling Firebase Auth.
}
```

### Without custom user data

When you don't require custom user data and it's sufficient to use `auth.UID`,
simply skip it in the return type:

```go
import "encore.dev/beta/auth"

// AuthHandler can be named whatever you prefer (but must be exported).
//encore:authhandler
func AuthHandler(ctx context.Context, token string) (auth.UID, error) {
    // Validate the token and look up the user id,
    // for example by calling Firebase Auth.
}
```

## Using auth data

Once the user has been identified by the auth handler, the API handler is called
as usual. If it wishes to inspect the authenticated user, it can use the
`encore.dev/beta/auth` package:

- `auth.Data()` returns the custom user data returned by the auth handler (if any)
- `auth.UserID()` returns `(auth.UID, bool)` to get the authenticated user id (if any)

For an incoming request from the outside to an API that uses the `auth` access level,
these are guaranteed to be set since the API won't be called if the auth handler doesn't succeed.

<Callout type="important">

If an endpoint calls another endpoint during its processing, and the original 
does not have an authenticated user, the request will fail. This behavior
preserves the guarantees that `auth` endpoints always have an authenticated user.

</Callout>

Note that the auth handler is invoked for **all** requests that specify an auth token,
which lets users optionally authenticate to public APIs. This can be useful in some
circumstances to return additional information. The second return value from `auth.UserID()`
can be used to determine if the request has an authenticated user.


Knowing what's what and who's who

Authenticating users


When using Encore to build applications you create one or more *environments*.
Each environment is an isolated, fully working instance of your backend.

With Encore you can create as many or as few environments as you wish,
all with the click of a button.

## Creating environments

To create an environment for your app, go to the **Environments** page,
and click `Create env` in the top right.

There you can decide on a name, whether it's a production environment
or a development environment (see [Environment Types](#environment-types) below).

Decide how you would like to deploy to the environment (either on pushing
to a Git branch or manually triggered).

Finally, select which cloud provider to deploy to (see [Cloud Providers](#cloud-providers) below),
and click `Create`. That's it!

## Environment Types

Encore offers two types of environments: **Production** and **Development**.
They differ in the type of infrastructure that is provisioned.

Production environments are provisioned for maximum reliability, availability and scalability.
Databases are provisioned as proper, managed databases with automatic backups.
Your backend code runs with auto-scaling to match your traffic requirements.

Development environments are provisioned for simplicity, cost efficiency and speed.
The databases are provisioned with persistent disks using Kubernetes, to offer
reasonable durability and scalability, suitable for the most development needs.

## Cloud Providers

Encore supports deploying your application to any of the major cloud providers,
as well as using Encore's own cloud (internally deployed using AWS), using your own cloud account.

This gives you enormous flexibility, letting you use Encore for improving your productivity
while maintaining the existing trust relationship you have with your cloud provider of choice.
This functionality also lets you easily deploy a hybrid or multi-cloud application, if desired.

<Callout type="important">

Note that Encore currently provisions a managed Kubernetes cluster when deploying to an external
cloud provider, which means the baseline costs are higher than when using Encore Cloud.

If you are evaluating Encore or aren't ready to scale to real production traffic yet,
we recommend starting with an Encore Cloud environment and later deploying an environment
with one of the major cloud providers.

</Callout>

### Provisioning infrastructure

When deploying to an external cloud, Encore will add a preliminary deployment phase
to provision the necessary infrastructure based on what your app needs.
This is computed with static analysis using the [Encore Application Model](/docs/concepts/application-model).

For certain infrastructure resources, you may be asked to tell Encore a bit more about the performance requirements
you have. This lets Encore provision appropriately sized infrastructure for your needs.
*(This is only necessary the first time you add a new infrastructure component, and only for Production environments.)*

Single cloud, multi cloud, or hybrid

Environments


All developers agree API documentation is great to have.
But the effort of maintaining inevitably leads to them becoming stale and out of date.

Encore provides the best of both worlds &mdash; great API docs that never become out of date &mdash;
by leveraging the [Encore Application Model](/docs/concepts/application-model) to determine the structure of every API.

![API Docs screenshot](/assets/img/api-docs-screenshot.png)

## Local development

You can also view API docs when developing locally.
When running your app with `encore run`, simply open up the API URL in your browser (by default `http://localhost:4060`).
You'll see the *local development dashboard* which gives you access to
request traces, logs and API docs for local development, right at your fingertips.

We did it so you don't have to

Generated API Docs

Encore Application Model


Encore applications are secure by default.

We've carefully designed the framework to make building secure applications
more convenient than insecure ones.

For example, Encore's [secret management](/docs/how-to/secrets) provides
an incredibly easy way of using secret keys, while at the same time providing
state of the art security behind the scenes, backed by HashiCorp Vault.

All communication between Encore and running applications leverage
mutual TLSv1.3 connections, and all database access is similarly encrypted
with certificate validation and strong security credentials.

Production databases provisioned through Encore with major cloud providers
provide managed, daily backups.

Industry standard best practices

Security


This section provides how-to guides for how to use Encore
to solve common backend development challenges.

- [Change your database schema](/docs/how-to/change-db-schema)
- [Receive Webhooks](/docs/how-to/webhooks)
- [Integrate with a frontend](/docs/how-to/integrate-frontend)
- [Use Firebase Auth with your app](/docs/how-to/firebase-auth)
- [Store API keys and secrets](/docs/how-to/secrets)
- [Integrate your app with GitHub](/docs/how-to/github)
- [Bring your own cloud](/docs/how-to/own-cloud)

How-to guides


Encore database schemas are changed over time using *migration files*.

Each migration file has a sequence number, and migration files are run
in sequence when deploying. Encore tracks which migrations have already run
and only runs new ones.

To change your database schema, add a new migration file using the next
available migration number.

For example, if you have two migration files already,
the next migration file should be named `3_something.up.sql` where
`something` is a short description of what the migration does.

<Callout type="important">
Database migrations are applied before the application is restarted
with the new code. Always make sure the old application code works with
the new database schema, so that things don't break while your new code
is being rolled out.
</Callout>

## Example

Let's say you have a single migration file that creates a `todo_item` table:

**`todo/migrations/1_create_table.up.sql`**
```sql
CREATE TABLE todo_item (
    id BIGSERIAL PRIMARY KEY,
    title TEXT NOT NULL,
    done BOOLEAN NOT NULL
);
```

And now you want to add a `created` column to track when each todo was created.
Add a new file:

**`todo/migrations/2_add_created_col.up.sql`**
```sql
ALTER TABLE todo_item ADD created TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW();
```

The next deploy Encore will notice the new migration file and run it, adding
a new column.

Change SQL database schema


Encore makes it easy to define APIs and expose them, but it works best when you are in charge of the API schema.

When you want to accept webhooks – other services that make an API call when something happens on their side –
they define the API schema that you must fulfill. They also often require you to parse custom HTTP headers and do
other low-level things that Encore usually lets you skip.

For these circumstances Encore lets you define **Raw Endpoints**. Raw endpoints operate at a lower abstraction level,
giving you access to the underlying HTTP request.

To define a raw endpoint, change the `//encore:api` annotation and function signature like so:

```go
package service

import "net/http"

// Webhook receives incoming webhooks from Some Service That Sends Webhooks.
//encore:api public raw
func Webhook(w http.ResponseWriter, req *http.Request) {
    // ... operate on the raw HTTP request ...
}
```

Like any other Encore API endpoint, this will be exposed at the URL <br/>
`https://<app-id>.encoreapi.com/<env>/service.Webhook`.

If you're an experienced Go developer, this is just a regular Go HTTP handler.

See the <a href="https://pkg.go.dev/net/http#Handler" target="_blank" rel="nofollow">net/http documentation</a>
for more information on how Go HTTP handlers work.


Call me maybe!

Receive webhooks


When developing with Encore, one of the great parts is that calling API endpoints
is just as easy as calling a function, and a great developer experience with immediate
auto-completion and compile-time checking of the data you pass in.

However, calling APIs from another project like a website or a mobile app has traditionally been
a frustrating experience, involving lots of manual boilerplate to create type-safe client classes.

Not anymore. With Encore, you can generate a type-safe client to get all the same benefits for an external projects.

## Generating API clients

To generate a client, simply use `encore gen client --lang=<lang> app-id`, where `lang` is the language code.
For example, to generate a type-safe TypeScript client, use `--lang=ts`.

The precise structure of the generated code depends on the language, to make sure it's idiomatic and easy to use,
but always includes all the publicly accessible endpoints, data structures, and documentation strings.

See `encore gen client --help` for a list of currently supported languages.

Integrate with a web frontend


Encore's [authentication support](/docs/concepts/auth) provides a simple yet powerful
way of dealing with various authentication scenarios.

<a href="https://firebase.google.com/docs/auth" target="_blank" rel="nofollow">Firebase Authentication</a>
{" "}is a common solution for quickly setting up a user store and simplifying social logins.

Encore makes it really easy to integrate with Firebase Authentication on the backend.

## Set up auth handler

First, install two modules:

```bash
$ go get firebase.google.com/go/v4 go4.org/syncutil
```

Next it's time to define your [authentication handler](/docs/concepts/auth).
It can live in whatever service you'd like, but it's usually easiest
to create a designated `user` service.

Create the `user/user.go` file and add the following skeleton code:

```go
package user

import (
	"context"
	"strings"

	"encore.dev/beta/auth"
	firebase "firebase.google.com/go/v4"
	fbauth "firebase.google.com/go/v4/auth"
	"go4.org/syncutil"
	"google.golang.org/api/option"
)

// Data represents the user's data stored in Firebase Auth.
type Data struct {
	// Email is the user's email.
	Email string
	// Name is the user's name.
	Name string
	// Picture is the user's picture URL.
	Picture string
}

// ValidateToken validates an auth token against Firebase Auth.
//encore:authhandler
func ValidateToken(ctx context.Context, token string) (auth.UID, *Data, error) {
    // TODO ...
}
```

## Initialize Firebase SDK

Next, let's set up the Firebase Auth client. We'll use
&nbsp;<a href="https://pkg.go.dev/go4.org/syncutil#Once" target="_blank" rel="nofollow">`syncutil.Once`</a>&nbsp;
to do it lazily the first time we need it.

Add to the bottom of our file:

```go
var (
	fbAuth    *fbauth.Client
	setupOnce syncutil.Once
)

// setupFB ensures Firebase Auth is setup.
func setupFB() error {
    return setupOnce.Do(func() error {
        opt := option.WithCredentialsJSON([]byte(secrets.FirebasePrivateKey))
        app, err := firebase.NewApp(context.Background(), nil, opt)
        if err == nil {
            fbAuth, err = app.Auth(context.Background())
        }
        return err
    })
}

var secrets struct {
	// FirebasePrivateKey is the JSON credentials for calling Firebase.
	FirebasePrivateKey string
}
```

## Validate token against Firebase

Now that we have the code to initialize Firebase Auth, we can use it from our `ValidateToken` auth handler.
Update the function to look like the following:

```go
func ValidateToken(ctx context.Context, token string) (auth.UID, *Data, error) {
    if err := setupFB(); err != nil {
		return "", nil, err
	}
	tok, err := fbAuth.VerifyIDToken(ctx, token)
	if err != nil {
		return "", nil, err
	}

	email, _ := tok.Claims["email"].(string)
	name, _ := tok.Claims["name"].(string)
	picture, _ := tok.Claims["picture"].(string)
	uid := auth.UID(tok.UID)

	usr := &Data{
		Email:   email,
		Name:    name,
		Picture: picture,
	}
	return uid, usr, nil
```

Great! We're done with the code. Now we just need to set up the secret.

## Set Firebase secret credentials

If you haven't already, set up a <a href="https://firebase.google.com" target="_blank" rel="nofollow">Firebase</a> project.

Then, go to **Project settings** and navigate to **Service accounts**.
Select `Go` as the language of choice and click `Generate new private key`.
Download the generated key and take note where it is stored.

Next, store the private key as your firebase secret.
From your terminal (inside your Encore app directory), run:

```bash
$ encore secret set --prod FirebasePrivateKey < /path/to/firebase-private-key.json
Successfully updated production secret FirebasePrivateKey
```

Now you should do the same for the development secret. The most secure way is to
set up a different Firebase project and use that for development.

Depending on your security requirements you could also use the same Firebase project,
but we recommend generating a new private key for development in that case.

Once you have a private key for development, set it similarly to before:

```bash
$ encore secret set --dev FirebasePrivateKey < /path/to/firebase-private-key.json
Successfully updated development secret FirebasePrivateKey
```

That's it! You can now call your Encore application and pass in Firebase tokens.
Encore will run your auth handler and validate the token against Firebase Auth.

Use Firebase Auth with your app


Encore applications are easy to integrate with GitHub for source code hosting.
To link your application to GitHub, head over to your app and open **App Settings &rarr; Git integration**
and follow the instructions.

For a more detailed guide on linking with GitHub, see the [Setup Walkthrough](#setup-walkthrough) below.

## Pull Requests & Preview Environments

Once you've linked with GitHub, Encore will automatically start building and running tests against
your Pull Requests.

Encore will also provision a dedicated Preview Environment for each pull request.
This environment works just like a regular development environment, and lets you test your changes
before merging.

Preview Environments are named after the pull request, so PR #72 will create an environment named `pr:72`.

### Frontend Collaboration

When you integrate Encore with a frontend, Preview Environments makes it really easy to collaborate
and test your changes against the frontend. Just update your frontend API client to point to the 
`pr:#` environment. This should just be a one-line change since your API client always specifies
the environment name like `https://<my-app>.encoreapi.com/<env>`.

If your pull request makes changes to the API, you can also easily [generate an API client](/docs/how-to/integrate-frontend)
against the new backend API using `encore gen client --env=pr:72 --lang=typescript my-app`

## Setup Walkthrough

Open your application on [app.encore.dev](https://app.encore.dev), and click
the app id in the top left. Select **App Settings** and then navigate to **Git Integration**.

First link your account with GitHub. Grant access either to all repositories or only the one(s)
you want to link with Encore.

When you come back to Encore, click the **Link app to GitHub repository** button:

<img class="max-w-lg w-full mx-auto rounded-lg shadow-lg border border-gray-100" src="/assets/img/git-begin.png" />

In the popup, select the repository you would like to link:

<img class="max-w-lg w-full mx-auto rounded-lg shadow-lg border border-gray-100" src="/assets/img/git-modal.png" />

Click **Link** and you're done:

<img class="max-w-lg w-full mx-auto rounded-lg shadow-lg border border-gray-100" src="/assets/img/git-linked.png" />

Integrate with GitHub


Encore supports deploying your application to any of the major cloud providers,
as well as using Encore's own cloud (internally deployed using AWS), using your own cloud account.

This gives you enormous flexibility, letting you use Encore for improving your productivity
while maintaining the existing trust relationship you have with your cloud provider of choice.
This functionality also lets you easily deploy a hybrid or multi-cloud application, if desired.

<Callout type="important">

Note that Encore currently provisions a managed Kubernetes cluster when deploying to an external
cloud provider, which means the baseline costs are higher than when using Encore Cloud.

If you are evaluating Encore or aren't ready to scale to real production traffic yet,
we recommend starting with an Encore Cloud environment and later deploying an environment
with one of the major cloud providers.

</Callout>

## Amazon Web Services (AWS)

To deploy to AWS, you must link your app to your AWS account.
Go to your app's settings page, and then click **Cloud Deploy**.

Follow the instructions to create an IAM Role, and then connect the role with Encore.
[Learn more in the AWS docs](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user.html).

<Callout type="warning">

For your security, make sure to check `Require external ID` and specify the
external ID provided in the instructions.

</Callout>

## Microsoft Azure

To deploy to Azure, you must link your app to your Azure organization.
Go to your app's settings page, and then click **Cloud Deploy**.
Follow the instructions to connect your account.

Note that currently you cannot link Encore with a personal Microsoft account.
To work around this, create a user using Azure AD (with at least `Cloud application administrator` permissions)
and use this user to link with Encore.

## Google Cloud Platform (GCP)

Deploying to GCP is actively in development and is available for preview. Let us know if you wish to use it
during the preview and we'll enable it for your application.

Better than your favorite beverage

Bring your own cloud

Reference

Engineering guides


Being able to divide a large application into smaller units is critical
to scaling, both in terms of scaling an engineering organization and
in terms of runtime performance. Crucially, we divide and conquer at *multiple levels*.

At the highest level, we divide an application into **systems**. A system represents
a group of related functionality, that share a common purpose from the product and business
perspective. When starting out you can just consider everything as part of a single system,
but as you grow it's useful to think in terms of a "user system", a "payment system" and similar.

Systems are divided into **services**, which represent the functionality grouped into a single
process in our backend. Within a service we have strong guarantees from running on a single CPU and
low latency access to memory. Between two services we define APIs as an interface for communication,
and these API calls happen over the network, which introduces milliseconds of latency and a larger
degree of unreliability.

Services are divided into **components**, which again represent smaller units of related functionality.
Between components we also define interfaces, but these interfaces are within a single runtime process
and therefore are much more reliable than a network call. For all intents and purposes we don't have to
worry about a component suddenly "disappearing".

Components are the lowest unit of division, and consist of **functions and data structures**.

## Domain driven design

While this structure appears somewhat rigid, it is only rigid in terms of naming conventions.
To successfully scale an engineering organization, it's critical that the top-level divisions (systems, services)
represent meaningful divisions in your product and business. As you grow, you will want to form
dedicated teams around aspects of your product, such as managing the user and login-related functionality
(the "user system").

We call this approach to designing your application **domain driven design**. We model our application
based on the domain we're operating in. If you are creating a Trello clone, for instance, it makes
sense to reason about your domain in terms of Boards, Columns and Cards.

## Dividing into services

It's become common to suggest that a service, in particular, should "do one thing and do it well".
However, in practice the only thing that makes the service boundary special is that it introduces
a network boundary in communication. Such a network boundary often makes it harder to make changes
that affect the API between two services.

Start instead by separating out different parts into components, and wait with splitting them
into different services until the API between them is fairly stable. Encore makes dealing 
with multiple services easier, but it's not magical and the network boundary is still real.
