Shipping Modern Systems

Code is the easy part. Shipping is the discipline.

A running collection of writing, courses, and tutorials on shipping software: version control, CI/CD, infrastructure-as-code, deployment, container builds, local dev, database migrations, and feature flags. Less about pipelines than about practice. What to automate, what to verify, and how to ship more often without paying for it later.

This index covers how your code gets to production. For what to build with, see System Architecture. For where it runs, see Infrastructure & Hosting. For keeping it healthy once running, see Production Operations.

Best Practices & Discussions

Cross-cutting writings on shipping software well.

Continuous Delivery

Tutorial Jez Humble & David Farley

The canonical text on building reliable, rapid, low-risk software releases through automation and discipline.

Accelerate

Tutorial Forsgren, Humble, Kim · IT Revolution

Data-driven follow-up to Continuous Delivery; the research foundation behind the DORA metrics.

The DevOps Handbook

Tutorial Kim, Humble, Debois, Willis · IT Revolution

Comprehensive practitioner's guide to applying DevOps principles across the value stream.

The Phoenix Project

Discussion Kim, Behr, Spafford · IT Revolution

The narrative DevOps text. A novel that introduced a generation of engineers to flow, feedback, and learning.

Trunk-Based Development

Best Practices Paul Hammant

Canonical reference on trunk-based development: short-lived branches, fast review, continuous integration.

OpenGitOps Principles

Best Practices OpenGitOps · CNCF

The CNCF-backed spec defining what GitOps is and what it isn't. Four principles, vendor-neutral.

Conventional Commits

Best Practices Conventional Commits

Lightweight convention for commit messages that machines can parse and humans can scan.

Semantic Versioning

Best Practices Tom Preston-Werner

MAJOR.MINOR.PATCH and the contract a version number actually conveys. The default for shared libraries.

Continuous Integration

Best Practices Martin Fowler

Fowler's canonical 2006 essay defining CI: integrate often, automate the build, keep main releasable.

Technology Radar

Best Practices Thoughtworks

Quarterly opinionated assessment of tools, techniques, platforms, and languages by Thoughtworks practitioners.

State of DevOps Report (DORA)

Best Practices DORA · Google Cloud

Annual research linking delivery practices to org performance. The empirical backbone of modern delivery.

Tools by Category

Version control

Git is the protocol. The host is what matters in practice. GitHub for default, GitLab for self-host-friendly, Codeberg and Forgejo for the ideological alternatives. The choice mostly determines your CI/CD options and how much of your workflow lives in the same UI as your code.

  • Git The distributed version control protocol everything else is built on.
  • GitHub The dominant code-hosting platform. Default if you want the biggest CI/CD ecosystem.
  • GitLab Self-host-friendly Git host with first-party CI/CD, packages, and registries.
  • Codeberg Non-profit, community-run Forgejo instance for OSS projects.
  • Forgejo Self-hostable Gitea fork governed by a non-profit. The community-first alternative.
  • Gitea Lightweight self-hostable Git host. The smallest, fastest option to run yourself.
  • Bitbucket Atlassian's Git host. Strongest when your team is already on Jira and Confluence.

Pro Git Book

Tutorial git-scm.com

The free official Git book. The definitive learn-Git-from-scratch and Git-reference resource.

GitHub Docs

Best Practices GitHub docs

Documentation root for GitHub features: repos, Actions, packages, security, and Codespaces.

GitLab Documentation

Best Practices GitLab docs

Reference for GitLab repos, CI/CD, registries, security, and self-managed installs.

Getting Started with Codeberg

Tutorial Codeberg docs

Walks new users through account setup, repos, and Codeberg's community-driven Forgejo-based platform.

Forgejo User Guide

Best Practices Forgejo docs

Official user-facing docs for repositories, pull requests, issues, and collaboration on a Forgejo instance.

Gitea Documentation

Best Practices Gitea docs

Installation, usage, and administration of the self-hosted Git forge.

CI/CD platforms

CI runs on every push; CD ships what passed. The line between them blurs in modern platforms. GitHub Actions wins by default because most teams are already on GitHub. CircleCI and Buildkite remain strong standalone offerings. Dagger and Earthly are the new wave that runs the same pipeline locally and remotely, important if you've ever debugged a YAML file by force-pushing.

  • GitHub Actions Default CI/CD for GitHub repos. Largest marketplace; YAML-driven workflows.
  • GitLab CI GitLab's first-party pipelines. Tight integration with the GitLab feature set.
  • CircleCI Standalone CI/CD with strong caching, orbs, and Docker-native execution.
  • Buildkite Hybrid model: hosted control plane, self-hosted agents. Strong at scale.
  • Jenkins The classic OSS CI server. Massive plugin ecosystem; operational cost is real.
  • Drone (Harness) Container-native CI from Harness. Lightweight YAML pipelines, easy self-host.
  • Earthly Earthfile-based builds that run identically locally and in any CI.
  • Dagger Programmable CI/CD: write pipelines in real languages, run anywhere with a container runtime.
  • TeamCity JetBrains' CI/CD server. Polished UI, strong build-chain modeling, free for small teams.

GitHub Actions Quickstart

Tutorial GitHub docs

Create your first workflow, trigger it on push, and explore the marketplace of pre-built actions.

CircleCI Quickstart

Tutorial CircleCI docs

Connect a repo, commit a config.yml, and watch your first CircleCI build run.

Jenkins Guided Tour

Tutorial Jenkins docs

Official guided tour: install Jenkins and create your first Pipeline.

Jenkins User Handbook

Best Practices Jenkins docs

Comprehensive reference for installing, configuring, and operating Jenkins in production.

Drone Quick Start

Tutorial Drone docs

Stand up a Drone server and run your first .drone.yml pipeline.

Dagger Documentation

Tutorial Dagger docs

Write pipelines in Go, Python, or TypeScript and run them anywhere with a container runtime.

Infrastructure as Code (IaC)

IaC lets you describe infrastructure as text and commit it. Terraform is the lingua franca; OpenTofu is its OSS fork after the license change. Pulumi gives you real programming languages. CDK is AWS's TypeScript/Python wrapper around CloudFormation. Crossplane brings IaC patterns into Kubernetes itself. Pick by how much programmability you want, and how much surface area your team can review.

  • Terraform HashiCorp's IaC tool. The default declarative language for cloud resources.
  • OpenTofu OSS fork of Terraform after the BSL license change. Drop-in compatible.
  • Pulumi IaC in real programming languages (TypeScript, Python, Go, C#) instead of HCL.
  • AWS CDK AWS's TypeScript/Python wrapper around CloudFormation. AWS-only, deeply integrated.
  • Crossplane Manage cloud resources from inside Kubernetes via CRDs. The K8s-native IaC pattern.
  • Ansible Agentless config management and orchestration via SSH. Still the default for VM provisioning.

Terraform Tutorials

Tutorial HashiCorp Developer

Hands-on tutorials covering AWS, Azure, GCP, Kubernetes, and Terraform Cloud workflows.

OpenTofu Documentation

Tutorial OpenTofu docs

Install OpenTofu and run your first plan/apply against any supported provider.

Pulumi Get Started

Tutorial Pulumi docs

Pick a language and cloud, stand up your first stack, and deploy it via the Pulumi CLI.

Deployment & GitOps

GitOps means Git is the source of truth for what runs in production. ArgoCD and Flux are the Kubernetes-native canonical implementations. Helm is the package format underneath. For non-K8s deployments, the toolchain is more fragmented: Spinnaker for the heavyweight Netflix legacy, Tekton for the pipelines-as-code framework. The orchestration runtime itself is in Infrastructure & Hosting; this section is about how code gets there.

  • ArgoCD CNCF GitOps controller for Kubernetes. The canonical declarative deploy tool.
  • Flux CNCF GitOps toolkit for Kubernetes. Composable controllers; the other canonical option.
  • Helm The Kubernetes package manager. Templates, releases, and chart repositories.
  • Spinnaker Netflix-born multi-cloud continuous delivery platform. Heavy but battle-tested.
  • Tekton CNCF pipelines-as-code framework. Composable building blocks for CI/CD on K8s.
  • Skaffold Google's CLI for K8s dev loops. Watch, build, push, deploy on file change.

ArgoCD Getting Started

Tutorial ArgoCD docs

Install ArgoCD, register a Git repo as a source of truth, and deploy your first application.

Flux Get Started

Tutorial Flux docs

Bootstrap Flux into a cluster, point it at a Git repo, and reconcile your first deployment.

Helm Quickstart

Tutorial Helm docs

Install Helm, search the chart repository, and release your first chart into a cluster.

Get Started Using Spinnaker

Tutorial Spinnaker docs

Orientation for new Spinnaker users covering applications, pipelines, and deployment strategies.

Skaffold Quickstart

Tutorial Skaffold docs

Run skaffold dev against a sample Node app on minikube to see the build/deploy inner loop.

Container build tools

Building a container image is no longer just docker build. BuildKit gives you parallelism and better caching. Buildpacks (Cloud Native, Heroku) and Nixpacks auto-detect your stack. Ko builds Go binaries straight into images, no Dockerfile required. The right pick depends on whether you want Dockerfiles, declarative configs, or zero config at all. The container runtime itself is in Infrastructure & Hosting.

  • BuildKit Modern image builder for Docker/OCI. Parallel stages, better caching, secret mounts.
  • Cloud Native Buildpacks CNCF spec for building OCI images from source. Auto-detects language and dependencies.
  • Nixpacks Railway's Nix-backed alternative to buildpacks. Auto-detect with reproducible builds.
  • ko Build Go binaries directly into OCI images. No Dockerfile required.
  • Jib Google's Java-to-container builder. Maven and Gradle plugins; no Docker daemon.
  • Bazel Google's hermetic build system. The heavyweight choice when you want fully reproducible builds.

BuildKit on GitHub

Tutorial GitHub

Project README covering frontends, caching, and modern Dockerfile features.

Nixpacks Documentation

Best Practices Nixpacks docs

Official source-to-OCI-image builder docs. Note: the project is in maintenance mode; Railway directs new projects to Railpack.

Get Started with ko

Tutorial ko docs

Install ko, authenticate to a registry, and build a Go app into a container without a Dockerfile.

Jib Maven Plugin

Tutorial GitHub

Quickstart, configuration, and goals for containerizing Java apps without Docker.

Jib Gradle Plugin

Tutorial GitHub

Gradle plugin for building OCI images from Java projects directly from your build.

Bazel: Getting Started

Tutorial Bazel docs

Install Bazel and follow language-specific tutorials (C++, Java, Go, etc.) to set up your first workspace.

Intro to Bazel

Best Practices Bazel docs

Conceptual overview of how Bazel approaches builds, dependencies, and reproducibility.

Local development environments

Local dev is half the developer experience. Dev Containers run a containerized dev environment in any editor that speaks the spec. mise and asdf manage runtime versions. direnv handles per-directory env vars. Nix flakes give you fully reproducible environments at the cost of a steep learning curve. Devbox wraps Nix in friendlier abstractions. Tilt is the option for Kubernetes-aware local dev. Docker Compose is in Infrastructure & Hosting; most teams use it locally too.

  • Dev Containers Open spec for editor-agnostic containerized dev environments. The default in VS Code.
  • mise Polyglot runtime version manager (formerly rtx). asdf-compatible, faster, with task running.
  • asdf Multi-language runtime version manager with a wide plugin ecosystem.
  • direnv Per-directory environment variables. Auto-loaded when you cd into a project.
  • Nix / Nix Flakes Functional package manager and reproducible-environment system. Steep but powerful.
  • Devbox Jetify's friendlier Nix wrapper. Per-project shells with simple JSON config.
  • Tilt Live-update workflow for Kubernetes dev. Smart rebuilds and a unified status UI.

Dev Containers Specification

Best Practices containers.dev

The devcontainer.json spec that VS Code, JetBrains, GitHub Codespaces, and others implement.

mise Documentation

Tutorial mise docs

Install mise and start managing runtime versions, env vars, and tasks per project.

asdf Getting Started

Tutorial asdf docs

Install asdf, configure your shell, and add plugins for the language runtimes you need.

direnv

Tutorial direnv docs

Project landing page with the canonical .envrc walkthrough and shell hook setup.

direnv Installation

Tutorial direnv docs

Install direnv and wire it into bash, zsh, fish, or your shell of choice.

Learn Nix

Best Practices NixOS.org

Official learning hub pointing to nix.dev tutorials and the Nix reference manual.

Devbox Quickstart

Tutorial Jetify docs

Initialize a devbox.json, add Nix-powered packages, and enter an isolated dev shell in minutes.

Tilt Tutorial

Tutorial Tilt docs

Build a Kubernetes dev loop with live updates, smart rebuilds, and a status UI.

Database migrations

Schema changes are deploys with no rollback. Migration tools force you to encode every change as a forward-only script. Atlas is the modern declarative entrant. Goose and golang-migrate are the language-agnostic mainstays. Alembic, Flyway, and Liquibase are the long-running standards in Python and JVM ecosystems. The database itself is a System Architecture decision; this section is about safely changing its shape over time.

  • Atlas Declarative schema-as-code migrations. Plans diffs, validates with linting and CI checks.
  • Goose Lightweight Go migration tool. SQL or Go-based migrations; ergonomic CLI.
  • golang-migrate Language-agnostic CLI + Go library for SQL migrations across most databases.
  • Alembic SQLAlchemy's migration tool. The Python standard for relational schemas.
  • Flyway JVM-era migration mainstay. SQL-first, broad DB support, strong baseline workflow.
  • Liquibase Enterprise schema-change platform. Tracks change sets with rich rollback support.
  • Prisma Migrate Migrations for Prisma ORM. Generated from schema changes; tight TypeScript story.
  • sqlx migrate The migration CLI shipped with Rust's sqlx. SQL-first with compile-time query checks.

Atlas Documentation

Tutorial Atlas docs

Declarative schema migrations: plan diffs, validate with linting, and ship via CI.

pressly/goose

Tutorial GitHub

Canonical README for the Go migration tool: CLI usage, SQL and Go migration formats, embed examples.

Goose Documentation

Best Practices Goose docs

Drivers, library usage, and recommended migration workflows.

Alembic Tutorial

Tutorial Alembic docs

Set up Alembic with SQLAlchemy, generate your first revision, and apply it to a database.

Flyway Documentation

Tutorial Redgate / Flyway

Install Flyway and version-control your database schema with SQL-first migrations.

Liquibase Documentation

Best Practices Liquibase docs

Authoritative reference covering changelogs, contexts, rollbacks, and CI/CD integration patterns.

sqlx-cli

Tutorial crates.io

Install sqlx-cli and use sqlx migrate add/run to manage SQLx migrations in Rust projects.

Feature flags & progressive delivery

Feature flags decouple deploy from release. You can ship code today and turn it on for 1% of users tomorrow. LaunchDarkly is the heavyweight commercial player. Statsig and PostHog combine flags with analytics. Unleash, Flagsmith, and GrowthBook are the OSS contenders. OpenFeature is the emerging standard for vendor-neutral SDKs. The decision is mostly whether you need experimentation (A/B testing) or just safe rollouts.

  • LaunchDarkly The heavyweight commercial feature-flag platform. Deepest experimentation features.
  • Statsig Flags + experimentation + analytics in one stack. Generous free tier.
  • PostHog Open-source product analytics with first-class feature flags and experiments.
  • Unleash Open-source feature-flag service. Self-hostable, with a hosted offering.
  • Flagsmith OSS feature-flag and remote-config platform. Strong on multi-environment workflows.
  • GrowthBook Open-source A/B testing and feature flags built around your existing data warehouse.
  • ConfigCat Lightweight feature-flag service with a generous free tier and simple SDKs.
  • OpenFeature CNCF-incubating standard for vendor-neutral feature-flag SDKs. Not a service; a spec.

PostHog Feature Flags

Tutorial PostHog docs

Use PostHog's flags plus analytics plus experimentation in one stack.

Unleash Documentation

Tutorial Unleash docs

"Get up and running with Unleash in less than 5 minutes" with links to SDKs and tutorials.

Flagsmith Quick Start

Tutorial Flagsmith docs

Four-step tutorial: create a project and flag, import the JS SDK, fetch flags, and toggle behavior.

GrowthBook Quick Start

Tutorial GrowthBook docs

End-to-end walkthrough for feature flagging and experimentation with GrowthBook Cloud or self-host.

ConfigCat Getting Started

Tutorial ConfigCat docs

Add a flag in the ConfigCat dashboard and evaluate it from any of 20+ supported SDKs.

Creators to follow

Engineers consistently publishing on continuous delivery, DevOps practice, and deployment.

Frequently Asked Questions

Short answers grounded in the work of practitioners shipping real software.

What's the minimum CI/CD setup for a small team?

On-push: lint, test, build. On-merge-to-main: deploy to staging automatically. On-tag (or manual approval): deploy to production. Three pipelines, three triggers; that's the whole shape. Almost any platform (GitHub Actions, GitLab CI, CircleCI) can do this in well under a hundred lines of YAML. Resist the urge to add review apps, canaries, or environments-per-branch until the basic flow is reliable and fast for everyone on the team.

Source: Martin Fowler: Continuous Integration

When should I add IaC to my project?

Roughly when reproducing your infrastructure stops being a one-day task. If a new environment takes you a week of clicking in cloud consoles, IaC has already paid for itself. The other trigger is auditability: when you need to know who changed what in prod, plain SCM history beats a CloudTrail digest. Start with the resources that change most often (DNS, load balancers, IAM) and grow outward. You don't need to import everything on day one.

Source: HashiCorp: What is Infrastructure as Code?

Should I use Terraform or Pulumi?

If your team is fluent in TypeScript, Python, or Go and you want loops, conditionals, and real testing, Pulumi is the natural fit. If you want the largest ecosystem of modules, the most provider coverage, and the easiest hiring story, Terraform (or OpenTofu) is still the default. The license change pushed many OSS teams to OpenTofu; check whether your CI provider's Terraform Cloud integration matters before picking.

Source: Thoughtworks Radar: OpenTofu (Adopt)

Do I really need GitOps, or is kubectl apply enough?

kubectl apply scales to small teams and short feedback loops. GitOps starts paying off when you have more than one cluster, more than a handful of services per cluster, or audit requirements that demand a clear answer to "what's running and who put it there?". The four OpenGitOps principles (declarative, versioned, pulled, continuously reconciled) describe what you gain. If you don't need the gains yet, the operational cost is real.

Source: OpenGitOps Principles

How do I handle database migrations in CI/CD safely?

Three rules: migrations are forward-only; every migration is backward-compatible with the previous app version; and migrations run before the app version that depends on them is deployed. That means expand/contract: add the new column or table first, ship the app that reads both old and new, backfill, then drop the old shape in a later release. Atlas, Alembic, Flyway, and Liquibase all support this pattern; the discipline comes from your team, not the tool.

Source: Martin Fowler: Evolutionary Database Design

When does feature flagging actually pay off?

Roughly when you start wanting to deploy more often than you want to release. Flags decouple the two: ship the code today, turn it on for 1% of users next week, ramp to 100% when the metrics hold. If your release cadence is monthly, you probably don't need a flag service yet. If it's daily and you have real users, the small operational cost pays for itself the first time you can roll back a bad release without rolling back a deploy.

Source: LaunchDarkly: What are Feature Flags?

How do I think about deploy frequency without breaking things?

The DORA research is unambiguous: high-performing teams deploy more often AND have lower change-failure rates. The way they get there is small batch sizes, fast feedback, and automation of the boring parts. The path is not to deploy more by relaxing review; it's to make each deploy smaller and the rollback cheaper. Charity Majors's framing is the right one: deploys shouldn't be scary because each one carries less risk, not because you've removed risk by waiting.

Source: DORA Research

From Smarter Dev

Original writing coming.

Smarter Dev essays, walkthroughs, and short courses on shipping software will land here as they're written.

Join the Discord to be notified

Last updated