Over the past few years Terraform has become the status quo in the Infrastructure as Code (IaC) landscape. It lets developers and DevOps engineers define and provision their infrastructure through writing Terraform configuration files using the HashiCorp Configuration Language (HCL).

But should you use it in 2024? Let’s unpack the main use cases, the pros and cons of Terraform, and its implications on developer experience and cloud cost management.

What is Terraform?

At its core, Terraform is an open-source tool used to build, modify, and version control infrastructure. It supports several service providers, enabling the creation of customized deployments. Terraform operates on the principle of Infrastructure as Code (IaC), where infrastructure is declared as code. This means it can be shared, reviewed, and controlled just like any other software codebase.

Key Terraform Use Cases

1. Multi-Cloud Deployment: Terraform helps with deploying applications across multiple cloud providers. With it, you can use the same tool and similar syntax to manage resources in AWS, GCP, Azure, and many others.
2. Microservices Infrastructure: Terraform can manage the infrastructure requirements of a microservices architecture, efficiently provisioning resources for each individual service.
3. Managing Diverse Cloud Resources: Terraform isn't limited to server management. It can also handle databases, DNS entries, and most other resources offered by a cloud provider.

The Upside of Using Terraform

• 1. Provider Agnostic: Terraform supports an impressive range of Terraform providers, including major cloud providers and many other services, making it a versatile choice for IaC needs.
• 2. Modularity: Terraform promotes the use of Terraform modules, helping you create reusable components that can be shared across your organization.
• 3. Version Control Capabilities: Since infrastructure is represented as code, it can be versioned and tracked using conventional version control systems like Git.

The Drawbacks of Terraform

• 1. Drift in State Management: Terraform must keep track of the current state of your resources. Managing this state file, in a codebase entirely separate from the application code, can be problematic in a large team setting – especially as individuals will often circumvent the process by making changes directly in the cloud provider's console. This often causes "drift", meaning the configuration file does not match reality.
• 2. Learning Curve: Terraform uses a proprietary syntax, HashiCorp Configuration Language (HCL), for infrastructure configuration. This can be challenging for beginners and comes with a steep learning curve. Moreover, it requires a lot of experience with each cloud provider's services to ensure proper configuration.
• 3. Significant Effort in Environment Setup: Configuration needs to be manually replicated for different environments, which is often time-consuming and error-prone. The scope involved often leads to teams relying on 1:1 duplicates of production for dev and staging environments. This causes expensive over-provisioning of these environments, significantly increasing cloud costs.
• 4. Debugging Errors: Debugging and error handling can be complex, especially with large deployments.
• 5. Disconnect Between Developers and DevOps: Developers, typically not versed in HCL, are often forced to rely on DevOps to provision resources, often slowing down the development process.

Alternatives to Terraform

Encore

Encore is a modern alternative, aimed at teams looking for an integrated and developer-experience focused approach to backend development. Encore is purposefully designed to make it less complex to build event-driven and distributed systems, and solves for both the local dev experience and assists with deployment using robust and scalable services from AWS and GCP.

It works by providing a Backend SDK that lets developers declare infrastructure semantics as part of the application code. Encore then parses the code and creates a fully type-safe model of your application's microservices, APIs, and infrastructure requirements. Encore then automatically runs your local environment, deploys to temporary preview environments for testing, and provisions infrastructure and deploys to your cloud (AWS/GCP).

Code Examples

In the following examples we're using Encore's TypeScript SDK, you can also refer to the docs to see examples from Encore's SDK for Go.

Defining a service

With Encore you define a service by creating a folder and inside that folder defining an API within a regular TypeScript file. Encore recognizes this as a service, and uses the folder name as the service name. When deploying, Encore will automatically provision the required infrastructure for each service.

On disk it might look like this:

/my-app
├── encore.app          // ... and other top-level project files
├── package.json  
│
├── hello               // hello service (a folder)
│   ├── hello.ts        // hello service code
│   └── hello_test.ts   // tests for hello service
│
└── world               // world service (a folder)
    └── world.ts        // world service code

This means building a microservices architecture is as simple as creating multiple directories within your application.

Defining APIs

Encore lets you easily define type-safe, idiomatic TypeScript API endpoints.

It's simple to accept both the URL path parameters, as well as JSON request body data, HTTP headers, and query strings.

It's done in a way that is fully declarative, enabling Encore to automatically parse and validate the incoming request and ensure it matches the schema, with zero boilerplate.

To define an API, use the api function from the encore.dev/api module to wrap a regular TypeScript async function that receives the request data as input and returns response data. This tells Encore that the function is an API endpoint. Encore will then automatically generate the necessary boilerplate at compile-time.

In the example below, we define the API endpoint ping which accepts POST requests and is exposed as hello.ping (because our service name is hello).

// inside the hello.ts file
import { api } from "encore.dev/api"

export const ping = api(
  { method: "POST" },
  async (p: PingParams): Promise<PingResponse> => {
    return { message: `Hello ${p.name}!` };
  }
);

Creating a Pub/Sub topic

Publishers & Subscribers (Pub/Sub) let you build systems that communicate by broadcasting events asynchronously. This is a great way to decouple services for better reliability and responsiveness.

Encore's Backend SDK lets you use Pub/Sub in a cloud-agnostic declarative fashion. At deployment, Encore automatically provisions the required infrastructure.

The core of Pub/Sub is the Topic, a named channel on which you publish events.

Encore makes it very simple to create Topics. For example, here's how you create a topic with events about user signups:

import { Topic } "encore.dev/pubsub"

export interface SignupEvent {
    userID: string;
}

export const signups = new Topic<SignupEvent>("signups", {
    deliveryGuarantee: "at-least-once",
});

To publish an Event, call publish on the topic passing in the event object (which is the type specified in the new Topic<Type> constructor):

const messageID = await signups.publish({userID: id});

// If we get here the event has been successfully published,
// and all registered subscribers will receive the event.

// The messageID variable contains the unique id of the message,
// which is also provided to the subscribers when processing the event.

Key Features

• No boilerplate: Create APIs and microservices without any overhead.
• Built-in Distributed Tracing: Easily trace the path and performance of requests.
• Automating API Documentation & Architecture Diagrams: Automatically keep documentation and diagrams up-to-date.
• Built-in Secrets Management: Handle sensitive data securely.
• Cloud Infrastructure Automation: Automatically provision infrastructure in local, preview, and your cloud environment in GCP and AWS.
• Backend Primitives: Use primitives like microservices, SQL databases, Pub/Sub message queues, and Redis caches all without manual infrastructure configuration.
• Push to Deploy: Get a Vercel-like workflow for your backend application, and integrate with Vercel for full-stack preview environments.

Benefits

• Cross-cloud Support: Encore makes applications cloud-portable by default, and lets you deploy your application to both AWS and GCP without any code changes.
• Preview Environments: Encore provides automated Preview Environments for each Pull Request, making it simple to validate changes with faster feedback loops.
• Developer Experience: Encore focuses on simplifying development by removing boilerplate and unlike Terraform does not require you to manually write infrastructure configuration in a proprietary language.
• Built-in Distributed Tracing: Applications built with Encore are automatically instrumented with distributed tracing and key performance metrics, aiding in debugging and performance monitoring.
• Type-Safety: Offers end-to-end type safety, including infrastructure, reducing runtime errors and bugs.
• Standardization: Provides a uniform way of building microservices and APIs, and provisioning infrastructure, that follows best practices.

Limitations

• To get the most value from Encore you need to the Backend SDK, which depending on your current situation may involve making some changes to your codebase. See the migration guide for advice.

Suitable for

• Encore is most suitable for teams looking to develop production-ready applications that require robust scalability and maintainability out of the box.
• It's particularly relevant for those who want to streamline their development workflow with built-in tools for common tasks, ensuring that developers can focus on building features rather than configuring the environment.

When to consider Encore

• When you want to build event-driven systems using scalable infrastructure like Pub/Sub.
• When you want to build a scalable system deployed to battle-tested infrastructure on AWS/GCP like Kubernetes or Cloud Run.
• When you want to automate API documentation and Architecture Diagrams, and remove most microservies boilerplate.
• When you want built-in observability like Distributed Tracing, Metrics, and integrations with tools like Grafana and Datadog.

Try Encore

• Sign up here to take Encore for a spin.
• See this guide for advice on how you can best evaluate Encore depending on your situation and priorities.

AWS CloudFormation

AWS CloudFormation is an IaC service specifically designed for AWS. It uses YAML or JSON templates to create and manage AWS resources.

Benefits & Key Features

• AWS-Specific: Deep integration with AWS services.
• Template-Based Configuration: Reusable code components.

Limitations

• No Multi-Cloud Support: Confined to AWS, limiting cross-platform applications.
• Cloud Lock-in: CloudFormation only works with AWS and often leads to significant cloud lock-in effects which can lead to increased costs and reduced flexibility over time.
• Language: JSON and YAML might be less readable for some compared to Terraform's HCL.

Suitable For

Organizations that are heavily invested in AWS and require native integration and management of AWS resources.

Google Cloud Deployment Manager (CDM)

Google Cloud Deployment Manager is an IaC tool specifically for Google Cloud Platform, using YAML-based configuration.

Benefits

• Google Cloud Specific: Tailored for GCP, ensuring seamless integration.
• YAML-Based Configuration: Offers consistency and structure.

Limitations

• Limited to GCP: Does not support other cloud platforms.
• Cloud Lock-in: Google CDM only works with GCP and often leads to significant cloud lock-in effects which can lead to increased costs and reduced flexibility over time.
• Less Community Support: May not have as extensive community support as Terraform.

Suitable For

Organizations using Google Cloud Platform exclusively, looking for an integrated solution to manage their resources.

Azure Resource Manager

Azure Resource Manager is designed for Microsoft's Azure platform, offering robust management of Azure resources through JSON-based templates.

Benefits

• Azure-Specific: Deeply integrated with Azure's suite of tools and services.
• Structured Templates: Promotes reusable and maintainable code.

Limitations

• No Support for Multi-Cloud: Confined to Azure, hindering cross-platform application.
• Cloud Lock-in: Azure Resource Manager only works with Azure and often leads to significant cloud lock-in effects which can lead to increased costs and reduced flexibility over time.
• Complexity: Can be challenging to learn, particularly for those new to Azure.

Suitable For

Businesses committed to Azure, seeking a powerful tool to orchestrate and manage their Azure resources effectively.

Conclusion

Terraform undoubtedly brings several benefits to the table, from multi-cloud support to enforcing consistent environments. However, the manual and complex nature of its configuration, the gap between application developers and DevOps, and cost considerations may make alternatives worth considering. Understanding the trade-offs is essential in making an informed choice that suits your project's needs and budget constraints.