git

Safely Storing Secrets in Git

Updated: 2018-11-05

I've worked on many projects where sensitive information is stored in a git repository. This includes SSH keys, SSL private keys, API keys, passwords, and client secrets. This practice is easily labelled Not Good.

You might be thinking, "Why should I worry? Our GitHub repository is private, and only our team can access the secrets!"

A private repository on GitHub or GitLab may seem like a safe storage area, since access is limited to a small set of specified parties. However, many teams don't think twice about allowing third party services to access these private repositories. Once you've enabled access, these services now have permission to read your secrets. Even if the service isn't malicious, thereis the chance that there is a vulnerability in the services that could be exploited to expose the files. Or perhaps there is a configuration flaw in your build server which grants everyone on the internet read access to your projects.

Storing secrets in our repositories is alluring because of its convenience, and I found myself recently tempted by this practice. I spent time investigating solutions to the secret storage problem and came across three easily incorporated projects:

  1. BlackBox
  2. git-secret
  3. git-crypt

I currently use BlackBox, as it was the first solution I discovered. However, I will be migrating to git-crypt because it offers a superior workflow with little user involvement.

BlackBox

BlackBox was created by Stack Overflow, initially to support secret storage for use by Puppet. Unlike the other two solutions I'll discuss, BlackBox is the only one that supports a multiple version control systems.

Protected files are encrypted with the public keys of all trusted users. If access needs to be revoked, delete the public key and re-encrypt the files.

BlackBox can encrypt and decrypt individual files. If you are using Puppet, you can also encrypt/decrypt individual strings.

I recommend BlackBox for developers who do not use git or work with a variety of version control systems.

git-secret

git-secret is a bash script based on gpg. git-secret allows you to encrypt individual files for storage inside of a git repository. git-secret provides commands to encrypt secret files before pushing to the server and to decrypt them for local use.

Protected files are encrypted with the public keys of all trusted users. If access needs to be revoked, delete the public key and re-encrypt the files. They will no longer be able to decrypt the secrets.

You can install git-secret with apt, yum, and brew.

git-crypt

git-crypt is a program that operates similarly to git-secret. Files that should be encrypted by git-crypt are tracked in the .gitattributes file, similarly to git-lfs.

Access permissions can be managed by adding individual gpg keys, similarly to git-secret. git-crypt also allows you to export a symmetric key which can be provided to collaborators.

Aside from an initial unlock command that needs to be used after cloning the repository, git-crypt encryption and decryption operations happen transparently. I find this workflow to be superior to git-secret and BlackBox.

You can install git-crypt through brew if you are on OSX. Otherwise you can manually compile git-crypt for your system.

Further Reading

Change Log

  • 20181105:
    • Fixed git-crypt and git-secret links

Code Cleanup: Splitting Up git Commits In the Middle of a Branch

I tend to follow a fairly traditional git development flow. When I am working on a new feature or a bug fix, I will make a branch and commit changes as I progress. I often just quickly commit incremental changes and make brief notes on my logic for that chunk.

I like to clean up these commits prior to merging the changes or submitting a pull request. I take the time to rebase the branch and squash related changes together until I am left with a set of atomic commits.

Sometimes I end up in a sticky situation where I have a single commit that contains changes destined for multiple atomic commits. Perhaps I added multiple files accidentally by using git commit -am, or perhaps I didn't originally notice that changes within a file would eventually be separated. If you catch this early you can revert a commit and split up the changes, but the situation is a little trickier if the offending commit is buried in the middle of a branch with other changes built on top of it.

After handling commit splits multiple times in the past two weeks, I figured I'd document a workflow.

Workflow for Splitting git Commits

Here's how I approach splitting up a git commit buried in the middle of a branch:

  1. Checkout the branch that you want to modify (e.g. pj/feature)
  2. Start an interactive rebase which includes your commit.
    • At a minimum, git rebase -i commit^ will start the rebase at the commit you want to split
    • You can also rebase the whole branch, which I usually do to split all target commits in one go
  3. Mark the commit(s) that you want to split with edit
  4. When git presents the commit that you want to split:
    1. Reset state to the previous commit using git reset HEAD^
    2. Use git add to carefully and incrementally add changes to the index
    3. Run git commit when you have a set of atomic changes that you are ready to commit
    4. Repeat the git add and git commit process until you have processed each set of changes represented by the commit
  5. Run git rebase --continue to resume or finish the rebase

That's all there is to it!

Further Reading

A GitHub Issue Template for Your Projects

Updated: 2019-03-02

Now that we have some example pull request templates, we might as well create an issue template for our GitHub projects.

Table of Contents:

  1. Creating an Issue Template for Your Project
  2. My Template
  3. Example Template File
  4. Further Reading

Creating an Issue Template for Your Project

Similarly to a PR template, it's super easy to setup an issue template for your project. Create a file named ISSUE_TEMPLATE and place it one of three locations:

  • The root of your project
  • .github folder
  • docs folder

An extension is optional, and GitHub supports Markdown (.md). I've chosen to create a Markdown template so you can use features like Markdown formatting, @-mentions, and task lists.

Once GitHub detects the ISSUE_TEMPLATE file, it will auto-populate new issues with the contents.

My Template

My issue template is much simpler than my PR template. I try to prompt users for the basic information I always need for new issues:

  • What behavior were you expecting?
  • What actually happened?
  • How can I reproduce it?
  • What was your environment like?
    • Firmware versions, host environment, hardware versions
  • Do you have any logs?

I've also added a Prerequisites section for projects with external issue contributors. This section prompts users to run through some basic steps before filing their issues.

If you only have private projects, feel free to remove the Prerequisites section (as I do for my own projects).

# Prerequisites

Please answer the following questions for yourself before submitting an issue. **YOU MAY DELETE THE PREREQUISITES SECTION.**

- [ ] I am running the latest version
- [ ] I checked the documentation and found no answer
- [ ] I checked to make sure that this issue has not already been filed
- [ ] I'm reporting the issue to the correct repository (for multi-repository projects)

# Expected Behavior

Please describe the behavior you are expecting

# Current Behavior

What is the current behavior?

# Failure Information (for bugs)

Please help provide information about the failure if this is a bug. If it is not a bug, please remove the rest of this template.

## Steps to Reproduce

Please provide detailed steps for reproducing the issue.

1. step 1
2. step 2
3. you get it...

## Context

Please provide any relevant information about your setup. This is important in case the issue is not reproducible except for under certain conditions.

* Firmware Version:
* Operating System:
* SDK version:
* Toolchain version:

## Failure Logs

Please include any relevant log snippets or files here.

Example Template File

You can find an example implementation file in templates GitHub repository. The ISSUE_TEMPLATE_example.md file contains the example template shown above. To use this file in your own project, copy that it into your project. Make any adjustments to these templates to suit your needs.

Further Reading

Change Log

  • 20190302:
    • Improved grammar
    • Added table of contents
    • Updated links to point to template repository

Related Articles