One Config to Rule Them All

So many configuration files

Every company I've worked for had dozens of repositories written in the same tech stack (either due to a micro-services architecture, or just many different types of projects, think a few backend applications, a CLI app, an open source project, etc...).

What all of these repositories had in common was a project-level configuration that was almost the same. Some examples are:

A CI/CD configuration: GitHub's .github/workflows/..., CircleCI's .circleci/config.yaml
An IDE configuration: VSCode's .vscode/ or Intellij IDEA's .idea/
A common linter/formatter configuration: .eslintrc, .prettierrc, .golangci.yaml, .pylintrc, ...
package.json scripts, .github/dependabot.yaml, .github/pull_request_template.md, Makefile

And the list goes on...

Whenever I wanted to make a change to such a configuration file, I had to open 10 or more pull requests in all of the different repositories, carefully copy-pasting the same changes, trying to stay focused (and sane) in the process to make sure I got everything changed correctly.

(Image created by this awesome bart chalkboard generator)

If we wanted to make 5 changes in 5 configuration files across 10 repositories, that would require 250 changes to be made just to update a configuration change!!!

Don't Repeat Yourself

One of the first principles we learn as programmers is Don't Repeat Yourself (DRY).

So how come we let ourselves rewrite the same configuration again and again in all of our projects?

Call me Sauron, but this drove me insane, and so I embarked on a quest to solve it.

Existing solutions

I'm not the first to tackle this problem and approach solving it. While looking for solutions, I found 2 interesting projects. Here are my opinionated thoughts on both of them:

mrm

A powerful tool that uses tasks written in javascript to maintain updates on shared config files intelligently.

👍🏼 Pros:

Powerful
Comes with predefined presets

👎🏼 Cons:

Complex and implicit (I don't want to write code if I don't have to)
Very oriented to the javascript ecosystem
Doesn't support syncing multiple repositories simultaneously
Doesn't support creating pull requests for the updates

cruft

A Cookiecutter compatible tool that supports initializing and updating project boilerplate.

A Cookie cutter is a tool that helps with creating new projects based on boilerplate templates.

👍🏼 Pros:

Builds on many good boilerplates (I personally don't like boilerplates, there usually isn't a one-size-fits-all and I like to get my hands dirty to make sure I understand the configuration)

👎🏼 Cons:

Doesn't support syncing multiple repositories simultaneously
Doesn't support creating pull requests for the updates
Has to be based on a cookie-cutter template

My requirements

When starting to think about a solution that would fit my needs, I had a couple of requirements in mind:

🌵 Stay DRY - Write a configuration once, and have it synced across many projects.
🤤 Keep It Simple Stupid (KISS) - Treat configuration snippets as simple text, not assuming anything about structure.
🙆🏻‍♀️ Allow flexibility, but not too much - Allow syncing whole files, or parts of them (currently, line-based).
🤖 Automate all the things - After an initial configuration, I want the solution to handle the rest.

Introducing Goplicate

Goplicate is a CLI tool that helps define common code or configuration snippets once and sync them to multiple projects.

(Made using gopherize.me)

It balances simplicity and flexibility by using comments to annotate blocks of configuration that we would like to sync. I've felt the 2 alternative solutions required too much of me to get started.

Disclaimer: I'm the author of Goplicate

Quick start in 4 simple steps

In the following simplified example, we'll sync an eslint configuration.

We'll end up having the following folder structure (full example here):

+ shared-configs-repo/
+   .eslintrc.js
  repo-1/
    .eslintrc.js
+   .goplicate.yaml
  repo-2/
    .eslintrc.js
+   .goplicate.yaml
  ...

Let's go!

1️⃣ Choose a config file that some of its contents are copied across multiple projects, and add goplicate block comments for the common-rules section of your desire:

repo-1/.eslintrc.js:

module.exports = {
    "extends": "eslint:recommended",
    "rules": {
+       // goplicate-start:common-rules
        // enable additional rules
        "indent": ["error", 2],
        "linebreak-style": ["error", "unix"],
        "quotes": ["error", "double"],
        "semi": ["error", "always"],
+       // goplicate-end:common-rules

        // override configuration set by extending "eslint:recommended"
        "no-empty": "warn",
        "no-cond-assign": ["error", "always"],
    }
}

2️⃣ Create a separate, centralized repository to manage all of the shared config files. We'll name it shared-configs-repo. Then, add a .eslintrc.js file with the common-rules snippet that we want to sync:

shared-configs-repo/.eslint.js:

module.exports = {
     "rules": {
          // goplicate-start:common-rules
          // enable additional rules
          "indent": ["error", 4],
          "linebreak-style": ["error", "unix"],
          "quotes": ["error", "double"],
          "semi": ["error", "always"],
          // goplicate-end:common-rules
    }
}

Goplicate snippets are simply the sections of the config file that we'd like to sync. In this example, we've also added the surrounding configuration to make it more readable, but it's not really needed.

3️⃣ Go back to the original project, and create a .goplicate.yaml file in your project root folder:

repo-1/.goplicate.yaml:

targets:
  - path: .eslintrc.js
    source: ../shared-configs-repo/.eslintrc.js

4️⃣ Finally, run goplicate on the repository to sync any updates:

This is only a simplified example. Let's see Goplicate's additional features.

Other supported features

A goplicate sync command to sync multiple repositories with a single command.
Support for checking out a new git branch, committing the changes, and opening a GitHub pull request (assuming GitHub CLI is installed and configured).
Template support using Go Templates with dynamic parameters.
Support for automatically running post hooks to validate that the updates worked well before opening a pull request.

Future Plans

Support for syncing from a remote source (a GitHub repository, etc...) to incorporate as part of a CI flow.
Support goplicating (😉) JSON files (they don't support comments so we'll have to do it another way).
Adding a documentation website with quick start examples for various use cases.
Adding CI integrations for the goplicate sync command to automate the opening of PRs for each update to the shared config repository.
Your next suggested feature 😁.

Goplicate is in the early stages of development - If you want to contribute, then feel free to open a PR or hit me up @ Ilai Fallach

Conclusion

If you're maintaining many different projects with similar config files or a single project with many such repositories, then try Goplicate out - It literally takes 4 steps to integrate into your existing projects' setup!

What do you think about Goplicate? Leave a comment and share your feedback and thoughts 🙏