Pre-Commit on ¬ just serendipity 🍀

pre-commit: authentication in internal github repos

serendipity@perrotta.dev (Thiago Perrotta) — Thu, 18 Jun 2026 14:42:26 +0200

♠ Problem statement: prek can’t clone a repo: that lives in an internal (private) GitHub repo — https:// clone fails without credentials.

I forked my personal pre-commit-hooks into an org-owned repo and updated all our .pre-commit-config.yaml files to point at it:

- repo: https://github.com/<org>/pre-commit-hooks
  rev: 5695e23285bef67d1dee957e5c12e86f58ba843b # frozen: v1.0.0
  hooks:
    - id: check-bash-shebang

This approach works locally (my GitHub token is broad).

CI (github actions) fails though:

fatal: could not read Username for 'https://github.com': terminal prompts disabled

The repo is INTERNAL, not public. As such, prek clones the repo: URL to fetch the hooks; with no credentials on the runner, it can’t.

First instinct: git@github.com:<org>/pre-commit-hooks. Runners have no SSH key though!

The fix is to keep https:// in the config, mint a short-lived token that can read the hooks repo, and tell git to rewrite the URL:

- name: Generate app token
  uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
  id: generate-token
  with:
    client-id: ${{ vars.APP_CLIENT_ID }}
    private-key: ${{ secrets.APP_PRIVATE_KEY }}
    owner: ${{ github.repository_owner }}
    repositories: pre-commit-hooks
    permission-contents: read
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
  with:
    persist-credentials: false
- name: Authenticate git for internal hook repos
  env:
    TOKEN: ${{ steps.generate-token.outputs.token }}
  run: git config --global url."https://x-access-token:${TOKEN}@github.com/".insteadOf "https://github.com/"
- uses: j178/prek-action@bdca6f102f98e2b4c7029491a53dfd366469e33d # v2.0.4

Two things worth noting.

First: never inline ${{ steps.generate-token.outputs.token }} directly into a run: shell string — zizmor (a GitHub Actions security linter) flags it as a template injection risk. Pass it via env: and use the shell variable instead.
Second: the repositories: pre-commit-hooks scope only works if the App is actually installed on that repo. That one is a manual step in GitHub’s UI — github.com/organizations/<org>/settings/installations → the App → Repository access → add the repo. No amount of YAML fixes it otherwise; it silently returns a 404 from the App installation endpoint otherwise.

After the installation grant, the clone goes through and prek runs cleanly.

🤖 Drafted with /bloggify.

— § —

Reply via email

#dev #git #pre-commit

★ Worktrunk

serendipity@perrotta.dev (Thiago Perrotta) — Mon, 25 May 2026 10:57:00 +0200

♠ Worktrunk:

Git worktree management for parallel AI agent workflows

I stumbled upon worktrunk (wt) due to a fortunate coincidence, serendipity at its best. I was pitching my own pancake wt to someone, and this person asked if I hadn’t heard of worktrunk.

If you look at its git star history¹, you’ll notice that it became popular only at the beginning of 2026, whereas I wrote my wt back in November 2025. I did proper diligence at the time but couldn’t find anything resembling my vision. Normally I prefer to reuse rather than writing from scratch.

“AI agent workflows” is a tagline that does not convince me. If anything, it actually worsens my appeal to try out the software. Seriously. Riding the wave is not helpful.

What really sold me was its usage examples. I was able to immediately tell that it had very similar goals to my own wt (sans AI).

Setup #

First, install worktrunk:

brew install worktrunk

It is as simple (and rusty!) as it could be:

% brew ls worktrunk
/opt/homebrew/Cellar/worktrunk/0.53.0/.crates.toml
/opt/homebrew/Cellar/worktrunk/0.53.0/.crates2.json
/opt/homebrew/Cellar/worktrunk/0.53.0/bin/wt
/opt/homebrew/Cellar/worktrunk/0.53.0/etc/bash_completion.d/wt
/opt/homebrew/Cellar/worktrunk/0.53.0/sbom.spdx.json
/opt/homebrew/Cellar/worktrunk/0.53.0/share/fish/vendor_completions.d/wt.fish
/opt/homebrew/Cellar/worktrunk/0.53.0/share/zsh/site-functions/_wt

Shell completion is a bonus that you’ll definitely want, as noted in a previous post:

# worktrunk: https://worktrunk.dev/
(( $+commands[wt] )) && eval "$(wt config shell init zsh)"

The snippet above integrates well with my own dotfiles; a vanilla setup that most people should follow is, instead, wt config shell install, which will populate your .bashrc / .zshrc accordingly with wt config shell init.

The shell integration alone makes it compelling enough to switch to wt, but we’re not done yet.

Workflow #

It is possible to customize wt. I value muscle memory and the ability to easily recall commands, therefore I crafted this basic configuration to facilitate the migration from my own wt:

worktree-path = "{{ repo_path }}/.worktrees/{{ branch | sanitize }}"

[aliases]
add = "wt switch --create {{ args }}"
bd = "wt remove -D {{ args }}"
cd = "wt switch {{ args }}"
checkout = "wt switch {{ args }}"
cleanup = "wt step prune {{ args }}"
co = "wt switch {{ args }}"
commit = "wt step commit {{ args }}"
create = "wt switch --create {{ args }}"
del = "wt remove {{ args }}"
delete = "wt remove {{ args }}"
goto = "wt switch --no-cd {{ args }}"
ls = "wt list {{ args }}"
new = "wt switch --create {{ args }}"
prune = "wt step prune {{ args }}"
rm = "wt remove {{ args }}"
tui = "wt switch {{ args }}"
world = "wt step prune {{ args }}"
xl = "wt list {{ args }}"

[pre-start]
prek = "test -f .pre-commit-config.yaml && prek install --install-hooks || true"

[pre-merge]
prek = "test -f .pre-commit-config.yaml && prek run --from-ref \"$(git symbolic-ref --short refs/remotes/origin/HEAD)\" --to-ref HEAD || true"

You can start by running wt config create and then iterating upon the newly generated config file, which is very well documented. What a great developer experience!

Let’s break it down.

We start with wt add <feature> (upstream: wt switch --create). This will create a worktree:

tperrotta ~/Workspace/perrotta.dev git:master!?
% wt add myfeature
◎ Running pre-start user:prek
  test -f .pre-commit-config.yaml && prek install --install-hooks || true
prek installed at `/Users/tperrotta/Workspace/perrotta.dev/.git/hooks/pre-commit`
✓ Created branch myfeature from master and worktree @ ~/Workspace/perrotta.dev/.worktrees/myfeature
  39.06s user 6.26s system 251% cpu 17.984 total

tperrotta ~/Workspace/perrotta.dev/.worktrees/myfeature git:myfeature ⎇ myfeature
% git st
## myfeature...master

At any time, we can check the current status with wt xl (upstream: wt list):

tperrotta ~/Workspace/perrotta.dev/.worktrees/myfeature git:myfeature ⎇ myfeature
% wt xl
  Branch     Status        HEAD±    main↕  Remote⇅  Path                    Commit    Age   Message
@ myfeature      _                                  ./.worktrees/myfeature  23df7c41  50m   name author explicitly
^ master      !? ^|      +5   -6              |     .                       23df7c41  50m   name author explicitly

○ Showing 2 worktrees, 1 with changes

When opening a new shell, we can easily switch to the newly created worktree from the repo root with wt cd (upstream: wt switch):

tperrotta ~/Workspace/perrotta.dev git:master!?
% wt cd myfeature
○ Switched to worktree for myfeature @ ~/Workspace/perrotta.dev/.worktrees/myfeature

tperrotta ~/Workspace/perrotta.dev/.worktrees/myfeature git:myfeature ⎇ myfeature
%

Once we are done (e.g. created/merged a pull request), we can delete the worktree with wt bd or wt del (upstream: wt remove -D).

That’s all! The core workflow steps are quite easy to assimilate.

You’ll notice that my worktrees are created in a .worktrees directory within the git repository root. This is due to my setting worktree-path in the config file. The default behavior of worktrunk is to use sibling directories (e.g. ../myrepo.myfeature from the git root).

As you can see from my config file, I have also integrated prek (pre-commit) to it:

the creation of a worktree yields prek install
the merge of a worktree (not covered in this post) yields prek run on the affected commits

wt has many other features; this is just the tip of the iceberg.

GitHub stars are a vanity metric but they are occasionally useful. ↩︎

— § —

Reply via email

#ai #bestof #dev #git #pkm #pre-commit

migrating from pre-commit to prek

serendipity@perrotta.dev (Thiago Perrotta) — Mon, 20 Apr 2026 10:00:00 +0200

♠ Previously.

Today I learned: prek is a drop-in replacement for pre-commit, rewritten in Rust. Same .pre-commit-config.yaml, same hook ecosystem, much faster.

A single binary with zero dependencies, much faster than the original pre-commit, drop-in compatible.

Install:

% brew install prek
# or
% cargo install prek

Then swap the command. Nothing else changes:

-pre-commit run --all-files
+prek run --all-files

Or skip the swap entirely: prek install -f drops in a shim that replaces the pre-commit binary, so existing scripts and git hooks keep working unchanged.

The CI workflow on this blog (prek.yml) now uses j178/prek-action instead of pre-commit/action, and the monthly autoupdate job (prek-autoupdate.yml) runs prek auto-update --freeze — the --freeze flag works the same.

The benchmark is what sold me. On cold caches, prek is around 10x faster at resolving and installing hooks; on warm runs, the difference is smaller but still noticeable.

Two other niceties pre-commit does not have:

Split configs across multiple files. prek supports multi-file configs, so hooks can live next to the subproject they belong to in a monorepo instead of piling up in a single top-level YAML.
Run against a commit or a range. prek run --from-ref A --to-ref B restricts the run to files changed between two refs — much faster than prek run -a because only the touched files are inspected. Example: lint everything on the current branch since it diverged from master:
```
% prek run --from-ref "$(git merge-base master HEAD)" --to-ref HEAD
```

The .pre-commit-config.yaml file stays put, and pre-commit.ci still reads it. No lock-in.

— § —

Reply via email

#dev #pre-commit

pre-commit: periodic garbage collection

serendipity@perrotta.dev (Thiago Perrotta) — Tue, 07 Apr 2026 10:56:21 +0200

♠ How funny, I do not remember setting this up. It is useful nonetheless:

thiago.perrotta ~
❯ crontab -l
30 12 1-7 * 5 [ "$(( ($(date +\%-d)-1)/7+1 ))" -eq 1 ] && /opt/homebrew/bin/pre-commit gc >> /Users/thiago.perrotta/.cronlogs/pre-commit-gc.log 2>&1

thiago.perrotta ~
❯ cat /Users/thiago.perrotta/.cronlogs/pre-commit-gc.log
2 repo(s) removed.
0 repo(s) removed.
3 repo(s) removed.
20 repo(s) removed.
10 repo(s) removed.
18 repo(s) removed.
2 repo(s) removed.
0 repo(s) removed.
0 repo(s) removed.
4 repo(s) removed.
61 repo(s) removed.
5 repo(s) removed.
0 repo(s) removed.
0 repo(s) removed.
0 repo(s) removed.
1 repo(s) removed.
26 repo(s) removed.
17 repo(s) removed.
9 repo(s) removed.
11 repo(s) removed.
1 repo(s) removed.
3 repo(s) removed.
10 repo(s) removed.
1 repo(s) removed.
6 repo(s) removed.
1 repo(s) removed.
6 repo(s) removed.
4 repo(s) removed.
0 repo(s) removed.
15 repo(s) removed.
0 repo(s) removed.
2 repo(s) removed.
4 repo(s) removed.
1 repo(s) removed.
16 repo(s) removed.
0 repo(s) removed.
0 repo(s) removed.
0 repo(s) removed.
0 repo(s) removed.
0 repo(s) removed.
0 repo(s) removed.

What does gc do? As per the docs:

Clean unused cached repos.

pre-commit keeps a cache of installed hook repositories which grows over time. This command can be run periodically to clean out unused repos from the cache directory.

This is on macOS. If I were on Linux, I’d prefer systemd timers instead.

Previously.

— § —

Reply via email

#dev #git #pre-commit

github workflow: external script

serendipity@perrotta.dev (Thiago Perrotta) — Sun, 22 Feb 2026 14:10:11 +0100

♠ Problem statement: execute a .js script in a GitHub workflow. The script must be stored in a separate file (than the workflow) in the same git repository.

Why store the file separately?

Inlining a big script is ugly and very non-elegant
No syntax highlighting when updating the script → error-prone
Risk of messing up with string interpolation or escaping characters → error-prone
Limited ability to use formatters and linters in inlined scripts

The following approach, taken from a real example, works well:

% tree {repo root}/.github/
.github/
├── scripts
│   └── drift-summary.js
└── workflows
    ├── drift-summary.yml
    [...]

name: 'Drift Summary'

on:
  schedule:
    - cron: '0 9 * * MON-FRI'
  workflow_dispatch:

permissions:
  issues: write

jobs:
  summarize:
    name: 'Update drift summary issue'
    runs-on: ubuntu-latest
    steps:
      - name: 'Checkout repository'
        uses: actions/checkout@1af3b93b6815bc44a9784bd300feb67ff0d1eeb3 # v6.0.0
        with:
          persist-credentials: false

      - name: 'Aggregate drift and update summary'
        uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7
        with:
          script: |
            const script = require('./.github/scripts/drift-summary.js');
            await script({ github, context });

Use github-script + require.

— § —

Reply via email

#dev #git #pre-commit

pre-commit: hazmat n1

serendipity@perrotta.dev (Thiago Perrotta) — Fri, 06 Feb 2026 18:28:16 +0100

♠ Problem statement: some CLI tools accept only a single file as an argument. Create a pre-commit local hook for them.

A common workaround is to wrap it in a bash for-loop (.pre-commit-config.yaml):

repos:
  - repo: local
    hooks:
      - id: my-formatter
        name: my-formatter
        language: system
        entry: bash -c 'for file in "$@"; do my-formatter fmt "$file"; done' --
        files: ^path/to/files/

This works, but it’s noisy and easy to get wrong. Plus it’s not elegant at all.

It turns out pre-commit has a built-in mechanism for this: hazmat n1. It runs the hook once per file, one at a time:

some hooks only take one filename argument. this runs them one at a time (which is super slow!)

repos:
  - repo: local
    hooks:
      - id: my-formatter
        name: my-formatter
        language: system
        entry: pre-commit hazmat n1 my-formatter fmt --
        files: ^path/to/files/

Note the trailing -- at the end of entry.

This is a super recent feature, added in pre-commit 4.5.0, in Nov 2025.

The name “hazmat” is supposedly intended to discourage people from using it. Yet, here we are.

“hazardous materials”

pre-commit provides a few entry prefix “helpers” for unusual situations.

in case it’s not clear, using these is usually a bad idea.

It is a bad idea, but sometimes the underlying hook / script simply does not support multiple arguments.

— § —

Reply via email

#dev #pre-commit

npm error code EBADF

serendipity@perrotta.dev (Thiago Perrotta) — Thu, 05 Feb 2026 13:21:00 +0100

♠ Symptom:

% pre-commit run -a
[INFO] Installing environment for https://github.com/rbubley/mirrors-prettier.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
An unexpected error has occurred: CalledProcessError: command: ('/Users/thiago.perrotta/.cache/pre-commit/repo78y8p1fk/node_env-system/bin/node', '/opt/homebrew/bin/npm', 'install', '-g', '/Users/thiago.perrotta/.cache/pre-commit/repo78y8p1fk/placeholder_package-0.0.0.tgz', 'prettier@3.8.1')
return code: 1
stdout: (none)
stderr:
    npm error code EBADF
    npm error errno EBADF
    npm error request to https://registry.npmjs.org/prettier failed, reason:
    npm error A complete log of this run can be found in: /Users/thiago.perrotta/.npm/_logs/2026-02-04T12_22_05_104Z-debug-0.log
Check the log at /Users/thiago.perrotta/.cache/pre-commit/pre-commit.log

Initially I thought it was an issue with prettier. The pre-commit repository is deprecated after all:

This is a fork of the (now archived) https://github.com/pre-commit/mirrors-prettier with the only difference being that it only references release versions of prettier, not e.g. alpha/beta versions.

The actual error was Little Snitch on macOS blocking all outgoing connections to the NPM registry.

How to address it?

Click “Manage Rules…”
Click “Deny” to list all network blocks
Delete the NPM registry entry

— § —

Reply via email

#dev #macos #pre-commit #security

new pre-commit hook: check-bash-shebang

serendipity@perrotta.dev (Thiago Perrotta) — Sun, 16 Nov 2025 20:48:17 -0300

♠ Previously.

I couldn’t resist creating a pre-commit hook for this:

repos:
  - repo: https://github.com/thiagowfx/pre-commit-hooks/
    rev: v0.0.11
    hooks:
      - id: check-bash-shebang
        types:
          - shell

In action:

% cd {pancake}
% pre-commit run -a check-bash-shebang
Check bash scripts use portable shebang..................................Failed
- hook id: check-bash-shebang
- exit code: 1

error: 'aws_china_mfa/aws_china_mfa.sh' uses '#!/bin/bash' instead of '#!/usr/bin/env bash'
error: 'aws_login_headless/aws_login_headless.sh' uses '#!/bin/bash' instead of '#!/usr/bin/env bash'
error: 'pritunl_login/pritunl_login.sh' uses '#!/bin/bash' instead of '#!/usr/bin/env bash'
error: 'sd_world/sd_world.sh' uses '#!/bin/bash' instead of '#!/usr/bin/env bash'
error: 'pdf_password_remove/pdf_password_remove.sh' uses '#!/bin/bash' instead of '#!/usr/bin/env bash'
error: 'img_optimize/img_optimize.sh' uses '#!/bin/bash' instead of '#!/usr/bin/env bash'
error: 'copy/copy.sh' uses '#!/bin/bash' instead of '#!/usr/bin/env bash'
error: 'ssh_mux_restart/ssh_mux_restart.sh' uses '#!/bin/bash' instead of '#!/usr/bin/env bash'
error: 'op_login_all/op_login_all.sh' uses '#!/bin/bash' instead of '#!/usr/bin/env bash'

— § —

Reply via email

#dev #pre-commit

go/keep-sorted

serendipity@perrotta.dev (Thiago Perrotta) — Mon, 22 Sep 2025 11:12:59 +0200

♠ Previously, previously, previously:

keep-sorted is, by far, one of my favorite tools to enforce tidying up a codebase.

Google Testing Blog, Sort Lines in Source Code.

They have finally released a TotT (“Testing on the Toilet”) episode about keep-sorted.

It’s such an underappreciated tool: not even 300 stars on GitHub. In the meantime, anything named “AI” or “Agents” or “LLM” rises to the top without any scrutiny of quality.

Now I am happy that I have somewhere to link to other than my blog.

— § —

Reply via email

#dev #pre-commit

pre-commit: autoupdate

serendipity@perrotta.dev (Thiago Perrotta) — Mon, 01 Sep 2025 12:05:52 +0200

♠ When using pre-commit, you can take advantage of its built-in pre-commit autoupdate command to keep your hook dependencies up-to-date.

There’s one major drawback of adopting it ubiquitously: it is virtually impossible to instruct it to skip certain repositories.

It wouldn’t be hard to update it to ignore hook dependencies with skip byline comments, but its author isn’t keen on implementing support for it.

So far I’ve been working around it by manually reverting undesired updates, but that is super tedious and inefficient.

Today I discovered a small utility, pre-commit-update, which explicitly supports that and was created precisely because of the limitation of the original subcommand:

pre-commit-update was created because there is no easy way to update your hooks by using pre-commit autoupdate as it is not versatile enough.

The tool (which is thankfully also written in Python, for uniformity) supports a config file to skip certain hooks from autoupdate.

— § —

Reply via email

#dev #pre-commit

"Fix typo"

serendipity@perrotta.dev (Thiago Perrotta) — Wed, 13 Aug 2025 11:13:35 +0200

♠ You, too, can make the ~~word~~ world a better place, one typo at a time.

The codespell project can be trivially adopted in a git repository via a pre-commit hook:

repos:
  - repo: https://github.com/codespell-project/codespell
    rev: 63c8f8312b7559622c0d82815639671ae42132ac # frozen: v2.4.1
    hooks:
      - id: codespell

— § —

Reply via email

#dev #pre-commit #serenity

★ check-json-schema: pre-commit hook

serendipity@perrotta.dev (Thiago Perrotta) — Fri, 11 Jul 2025 22:05:32 +0200

♠ Problem statement: given an entry.json file, validate it against a given schema.json file (for example, found via the schema store).

I like the check-jsonschema python package:

pipx run check-jsonschema --schemafile schema.json entry.json

What if we want to do the validation via a pre-commit.com hook?

repos:
  - repo: https://github.com/python-jsonschema/check-jsonschema
    rev: 0.33.2
    hooks:
      - id: check-jsonschema
        files: ^entry\.json$
        args:
          - --schemafile
          - schema.json

This approach works well, but it requires the upkeep of a mapping from JSON file to JSON schema.

If we had various JSON files within a single repository, this would result in a maintenance burden in the pre-commit config file.

For example:

repos:
  - repo: https://github.com/python-jsonschema/check-jsonschema
    rev: 0.33.2
    hooks:
      - id: check-jsonschema
        alias: check-jsonschema-entry1
        files: ^entry1\.json$
        args:
          - --schemafile
          - schema1.json
      - id: check-jsonschema
        alias: check-jsonschema-entry2
        files: ^entry2\.json$
        args:
          - --schemafile
          - schema2.json

It’s somewhat annoying to maintain multiple entries in that fashion. Furthermore it can easily get outdated: upon the deletion of entry1.json from the repository, you’d need to remember to manually delete its corresponding entry from the pre-commit config file.

Can we do better?

What if we embedded the schema reference within the JSON file itself?

For example, with a local schema:

{
  "$schema": "schema.json"
}

Or with a remote schema URI, via HTTPS:

{
  "$schema": "https://example.com/schema.json"
}

Now the JSON file is self-contained, and there’s no need to define a mapping from it to its JSON schema elsewhere.

How can we integrate this approach within pre-commit?

Initially I thought that check-metaschema from the check-jsonschema python package was the answer, but it turns out it isn’t.

In fact, this isn’t currently possible. There’s a feature request, filed ~2 years ago, with no consensus yet on how move forward. Its maintainer seems helpful and concerned with backwards compatibility.

For the sake of unblocking myself, I decided to wrap the ability to do so within my own pre-commit hook definition.

This gave birth to the following repository: thiagowfx/check-json-schema-meta, which I also announced in the original feature request thread.

Usage:

repos:
  - repo: https://github.com/thiagowfx/check-json-schema-meta
    rev: v0.0.2
    hooks:
      - id: check-json-schema-meta

It goes through each JSON file in the repository, looking for an optional top-level $schema key. When found, the file is validated against the defined schema.

A --strict command-line flag enforces the presence of the $schema key, otherwise the default behavior is to skip files that do not have it.

The underlying schema validation defers the work to the same python package first mentioned in this post. Because it uses SchemaLoader, both local and remote (http(s)) file references are supported.

This is the first original pre-commit hook I have written. By original I mean that I couldn’t find any pre-existing open source hook that accomplishes this task. The entire repository was developed with AI assistance (Claude Code), but I wasn’t satisfied with out-of-the-box defaults; I had to heavily modify it to shape it to my taste – which was achieved with a mix of follow-up prompting and local edits.

— § —

Reply via email

#bestof #dev #pre-commit

Helm repository with a trailing slash

serendipity@perrotta.dev (Thiago Perrotta) — Tue, 24 Jun 2025 11:32:50 +0200

♠ A repository URL with a trailing slash, and the same URL without a trailing slash, are treated as different repos. We found this out the hard way:

Failed to load target state: failed to generate manifest for source 1 of 2: rpc error: code = Unknown desc = error building helm chart dependencies: failed to add helm repository https://chartmuseum.{corp intranet}/orgs/{corp}/: failed to add repository: failed to get command args to log: `helm repo add https:--chartmuseum.{corp intranet}-orgs-{corp}- https://chartmuseum.{corp intranet}/orgs/{corp}/` failed exit status 1: Error: looks like "https://chartmuseum.{corp intranet}/orgs/{corp}/" is not a valid chart repository or cannot be reached: failed to fetch https://chartmuseum.{corp intranet}/orgs/{corp}/index.yaml : 401 Unauthorized

Let’s put aside the fact the error message REALLY likes to repeat the repo name.

This is a repository that we’ve already configured, with proper authentication credentials. Why would it start to fail now, all of a sudden?

You already got spoiled in the post title. Given Chart.yaml:

apiVersion: v2
name: corp-postgres-operator
description: Corp Postgres Operator
type: application
version: 1.1.13

dependencies:
  - name: postgres-operator
    repository: https://chartmuseum.{corp intranet}/orgs/archive

And also:

apiVersion: v2
name: corp-postgres-operator
description: Corp Postgres Operator
type: application
version: 1.1.13

dependencies:
  - name: postgres-operator
    repository: https://chartmuseum.{corp intranet}/orgs/archive/

These two dependencies end up being treated differently by Argo, even though they are effectively the same. It’s an upstream bug.

To work around it, I decided to simply remove all trailing slashes from helm repository dependencies. Canonicalize everything!

A git pre-commit.com hook is a good mechanism to do so:

repos:
  repo: local
  hooks:
    - id: helm-repo-trailing-slash
      name: Helm repositories must not contain trailing slashes in Chart.yaml
      files: Chart.yaml$
      language: pygrep
      entry: '\s*repository: [\w:/.]+/$'

Run pre-commit run -a helm-repo-trailing-slash, fix all violations, and then never think about this again.

Should the upstream bug ever be fixed, we can remove this workaround.

— § —

Reply via email

#dev #kubernetes #pre-commit

pre-commit: add hugo build

serendipity@perrotta.dev (Thiago Perrotta) — Sat, 24 May 2025 17:29:48 +0200

♠ Sometimes I make changes to this blog that yield build errors with its static site generator (hugo).

It’s trivial to spot these changes with a simple hugo build, but I don’t always run it.

Here’s what a typical error looks like:

% hugo build
Start building sites …
hugo v0.147.5+extended+withdeploy darwin/arm64 BuildDate=2025-05-22T11:37:19Z VendorInfo=brew

ERROR [en] REF_NOT_FOUND: Ref "nonexistent/2025-03-24-maccy-pin-items": "/Users/tperrotta/Workspace/perrotta.dev/content/posts/2025-05-16-espanso-hello-world.md:10:14": page not found
Total in 242 ms
Error: error building site: logged 1 error(s)

The same happens when invoking just, as you would expect:

% just build
hugo --environment production --gc --minify
Start building sites …
hugo v0.147.5+extended+withdeploy darwin/arm64 BuildDate=2025-05-22T11:37:19Z VendorInfo=brew

ERROR [en] REF_NOT_FOUND: Ref "nonexistent/2025-03-24-maccy-pin-items": "/Users/tperrotta/Workspace/perrotta.dev/content/posts/2025-05-16-espanso-hello-world.md:10:14": page not found
Total in 280 ms
Error: error building site: logged 1 error(s)
error: Recipe `build` failed on line 17 with exit code 1

Problem statement: catch such build errors at git pre-commit time.

This is trivial with a pre-commit.com hook:

Author: Thiago Perrotta <{email redacted}>
Date:   Sat May 24 17:26:59 2025 +0200

    pre-commit: add just build

diff --git .pre-commit-config.yaml .pre-commit-config.yaml
index 60eb2022..623e135d 100644
--- .pre-commit-config.yaml
+++ .pre-commit-config.yaml
@@ -112,5 +112,10 @@ repos:
         language: pygrep
         types:
           - markdown
+      - id: hugo
+        name: Hugo
+        language: system
+        entry: just build
+        pass_filenames: false

Here’s the end result:

% git commit -m "test"
Check hooks apply to the repository.............................................(no files to check)Skipped
Check for useless excludes......................................................(no files to check)Skipped
[...]
Forbid usage of UTM analytics / tracking............................................................Passed
Hugo................................................................................................Failed
- hook id: hugo
- exit code: 1

hugo --environment production --gc --minify
Start building sites …
hugo v0.147.5+extended+withdeploy darwin/arm64 BuildDate=2025-05-22T11:37:19Z VendorInfo=brew

ERROR [en] REF_NOT_FOUND: Ref "a/2024-12-23-unbuffer": "/Users/tperrotta/Workspace/perrotta.dev/content/posts/2025-05-24-watch-with-color.md:54:54": page not found
Total in 276 ms
Error: error building site: logged 1 error(s)
error: Recipe `build` failed on line 19 with exit code 1

— § —

Reply via email

#dev #pre-commit

pre-commit: make a hook opt-in

serendipity@perrotta.dev (Thiago Perrotta) — Thu, 27 Mar 2025 13:22:44 +0100

♠ Recently we added the following hook to our pre-commit setup:

repos:
  - repo: https://github.com/sbrunner/hooks
    rev: ee105bd276e163fce85744b39104ca563f923c47 # frozen: 1.4.0
    hooks:
      - id: helm-lock

It can be triggered with pre-commit run --all-files helm-lock.

It turned out to be an issue because not every developer has read-only access to our helm repositories.

The workaround was to make this hook opt-in.

We can do so by adding it to the manual stage:

repos:
  - repo: https://github.com/sbrunner/hooks
    rev: ee105bd276e163fce85744b39104ca563f923c47 # frozen: 1.4.0
    hooks:
      - id: helm-lock
        stages:
          - manual

It can be then invoked like this: pre-commit run --all-files helm-lock --hook-stage manual.

An alternative could have been to instruct people to set SKIP=helm-lock, however defaults are powerful.

— § —

Reply via email

#dev #pre-commit

pre-commit: pin dependencies with --freeze

serendipity@perrotta.dev (Thiago Perrotta) — Thu, 20 Mar 2025 18:12:57 +0100

♠ Today I learned: pre-commit autoupdate has a --freeze option that replaces git tags with SHAs (hashes) on upstream hooks (dependencies).

Here’s how to adopt them, using this blog as an example:

% pre-commit autoupdate --jobs $(nproc) --freeze
[https://github.com/lalten/check-gha-pinning] updating v1.3.0 -> v1.3.0 (frozen)
[https://github.com/google/keep-sorted] updating v0.6.0 -> v0.6.0 (frozen)
[https://github.com/sirosen/texthooks] updating 0.6.8 -> 0.6.8 (frozen)
[https://github.com/google/yamlfmt] updating v0.16.0 -> v0.16.0 (frozen)
[https://github.com/pre-commit/pre-commit-hooks] updating v5.0.0 -> v5.0.0 (frozen)
[https://github.com/shssoichiro/oxipng] updating v9.1.3 -> v9.1.4 (frozen)
[https://github.com/codespell-project/codespell] updating v2.4.1 -> v2.4.1 (frozen)
[https://github.com/gitleaks/gitleaks] updating v8.23.3 -> v8.24.0 (frozen)

The resulting diff (redacted one of the hooks for brevity):

diff --git .pre-commit-config.yaml .pre-commit-config.yaml
index 30aac4a2..841eac9b 100644
--- .pre-commit-config.yaml
+++ .pre-commit-config.yaml
@@ -23,7 +23,7 @@ repos:
       - id: check-hooks-apply
       - id: check-useless-excludes
   - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v5.0.0
+    rev: cef0300fd0fc4d2a87a85fa2093c6b283ea36f4b  # frozen: v5.0.0
     hooks:
       - id: check-executables-have-shebangs
@@ -41,27 +41,27 @@ repos:
           - -D .codespell-dictionary.txt
           - -I .codespell-ignore-words.txt
   - repo: https://github.com/gitleaks/gitleaks
-    rev: v8.23.3
+    rev: f565e4e10c6c8081e374235bb4e4bcb8c0c2fa63  # frozen: v8.24.0
     hooks:
       - id: gitleaks
   - repo: https://github.com/google/keep-sorted
-    rev: v0.6.0
+    rev: df93c2722b6126556183749880f16a9beb664bb4  # frozen: v0.6.0
     hooks:
       - id: keep-sorted
   - repo: https://github.com/google/yamlfmt
-    rev: v0.16.0
+    rev: ac76bb931851dda64a55ef746ca9b8bb41634b98  # frozen: v0.16.0
     hooks:
       - id: yamlfmt
   - repo: https://github.com/lalten/check-gha-pinning
-    rev: v1.3.0
+    rev: 225deee5dff16fae532978007ce6e37045e14c68  # frozen: v1.3.0
     hooks:
       - id: check-gha-pinning
   - repo: https://github.com/shssoichiro/oxipng
-    rev: v9.1.3
+    rev: bc8c36456858fe4678460bdea520a4ec0bc50dc4  # frozen: v9.1.4
     hooks:
       - id: oxipng
   - repo: https://github.com/sirosen/texthooks
-    rev: 0.6.8
+    rev: 13a42592dd28b6b688bb1c23fa769234984d4ca7  # frozen: 0.6.8
     hooks:
       - id: fix-ligatures
       - id: fix-smartquotes

This is useful to mitigate supply-chain attacks such as the recent tj-actions/changed-files one.

— § —

Reply via email

#dev #pre-commit

★ git freeze, git thaw

serendipity@perrotta.dev (Thiago Perrotta) — Wed, 19 Mar 2025 13:27:43 +0100

♠ Now I am becoming one of the snobs™ with the first sentence in the following paragraph. Here we go.

I had Claude Code¹ write me a pair of git utilities: git freeze and git thaw.

First of all: “to thaw” means “to defrost”:

To change from a frozen solid to a liquid by gradual warming.

Perhaps git unfreeze would have been more intuitive, let’s leave the bikeshedding to another day though.

What should these utilities do? They are inspired by the ones with the same name in Chromium depot_tools. Direct link:

git-freeze(1) allows you to put the current branch in ‘suspended animation’ by committing your changes to a specially-named commit on the top of your current branch. When you come back to your branch later, you can just run git-thaw(1) to get your work-in-progress changes back to what they were.

Installing depot_tools(7) just to get these two utilities is too bloated though. The security software at work flagged it as malware². And they didn’t even bother to package it for Linux distributions in the first place.

`git-freeze` #

#!/bin/sh
#
# git-freeze - Stage all files and create a git-freeze commit

set -e

# Check if we're inside a git repository
if [ "$(git rev-parse --is-inside-work-tree 2>/dev/null)" != "true" ]; then
  echo "Error: Not in a git repository"
  exit 1
fi

# Stage all changes (tracked and untracked)
git add -A

# Create a commit with the freeze message
git commit -m "git-freeze" -n

echo "Git freeze created successfully"

`git-thaw` #

#!/bin/sh
#
# git-thaw - Undo a git-freeze operation

set -e

# Check if we're inside a git repository
if [ "$(git rev-parse --is-inside-work-tree 2>/dev/null)" != "true" ]; then
  echo "Error: Not in a git repository"
  exit 1
fi

# Check if there's at least one commit
if ! git rev-parse --verify HEAD >/dev/null 2>&1; then
  echo "Error: No commits yet"
  exit 1
fi

# Check if the last commit is a git-freeze commit
if ! git log -n 1 --pretty=%B | grep -q "^git-freeze$"; then
  echo "Error: Last commit is not a git-freeze commit"
  exit 1
fi

# Undo the git-freeze commit
if git rev-parse --verify HEAD^ >/dev/null 2>&1; then
  # If there's a parent commit, reset to it
  git reset "HEAD^"
else
  # If it's the initial commit, delete the reference and unstage everything
  git update-ref -d HEAD
  git restore --staged .
fi

echo "Git freeze thawed successfully"

Post-AI cleanup #

The only change I had to make was to add -n to git commit in git-freeze so that pre-commit hooks would be skipped upon freeze.

Usage #

Within an existing git repository, do some work in a custom branch.
Decide that something else more important and/or urgent needs your attention. Let go of git worktrees in this instance.
git freeze
Switch to a new branch. Do whatever. Done.
git checkout <your previous branch>
git thaw
Resume your previous work. Profit.

Edit(2025-03-20): A friend asked that is the difference between git freeze and git stash. The main difference is that git stash does not handle files not yet tracked by git, whereas git freeze does. I think of freeze as the equivalent of completely clearing up my physical desk of all objects so that I can focus on something else.

An agentic coding tool made by Anthropic. Currently in beta as a research preview. ↩︎
In hindsight that’s not even surprising. ↩︎

— § —

Reply via email

#bestof #dev #git #pre-commit

YOLO: debug only after push

serendipity@perrotta.dev (Thiago Perrotta) — Fri, 14 Mar 2025 15:16:57 +0100

♠ Whenever writing new posts in this blog I rarely preview them in advance, especially when they are short and simple¹.

My rushed workflow is roughly:

open a new terminal tab / window
cd ~blog
just new "hello world"
vim content/posts, hit ‘G’, hit ‘k’, hit ‘ENTER’
draft the post
:Gwq (vim-fugitive, git integration)
git commit -m "new post: hello world"
git push

ONLY then, just, open web browser, see if it looks OK.

Does it NOT look OK (grammar errors, markdown issues, typos)? Make a quick edit, :Gwq again, git amend && git push -f.

Normally this process happens faster than the CI pipeline takes to build and subsequently publish the site.

No AI involved™, not even a spellchecker².

Which is what I strive for most of the time anyways. ↩︎
Not completely true: codespell is integrated via pre-commit. vim has :set spell. That’s all though. There’s no “make this sound better” or “rewrite this more professionally”. What you read is what comes straight off my brain. ↩︎

— § —

Reply via email

#dev #meta #pre-commit #serenity

keep-sorted, with quotes

serendipity@perrotta.dev (Thiago Perrotta) — Thu, 13 Mar 2025 13:00:06 +0100

♠ keep-sorted has been previously covered.

Problem statement: how to automatically sort the following YAML list?

- azure-workload-identity
- clustermon
- flatfile
- kube2iam
- mongo
- node-local-dns
- "pgbouncer*"
- "rabbitmq*"
- "redis*"
- traefik
- vault

I would naturally add keep-sorted to it, but the result is not what I expected:

# keep-sorted start
- "pgbouncer*"
- "rabbitmq*"
- "redis*"
- azure-workload-identity
- clustermon
- flatfile
- kube2iam
- mongo
- node-local-dns
- traefik
- vault
# keep-sorted end

Quotes are coming first!

Which does indeed make sense. They are ASCII characters (ascii(7)), value 34, which comes before a-z. Oh well.

I tried ignore_prefixes=" to no avail. I also tried by_regex=[\w-]+ to no avail.

Only the following form worked:

# keep-sorted start by_regex=['\w+[\w ]*']
- azure-workload-identity
- clustermon
- flatfile
- kube2iam
- mongo
- node-local-dns
- "pgbouncer*"
- "rabbitmq*"
- "redis*"
- traefik
- vault
# keep-sorted end

…but it’s a mouthful. I filed an upstream bug: https://github.com/google/keep-sorted/issues/76.

Edit(2025-01-04): Thanks Ho Man for the following suggestion:

# keep-sorted start by_regex=\w+
- azure-workload-identity
- clustermon
- flatfile
- kube2iam
- mongo
- node-local-dns
- "pgbouncer*"
- "rabbitmq*"
- "redis*"
- traefik
- vault
# keep-sorted end

— § —

Reply via email

#dev #pre-commit

commit, pre-commit, commit

serendipity@perrotta.dev (Thiago Perrotta) — Wed, 12 Mar 2025 16:41:53 +0100

♠ Premature optimization is the root of all evil but I run into this almost every single day, it’s time to finally address it.

Problem statement: When using pre-commit with a hook that will automatically fix an issue, the commit will fail the first time it runs. This means I need to issue two git commit -a commands in order to effectively create a commit: the first one will trigger pre-commit run, the second one will add the changes / fixes done by it. The question is: how can I merge these two steps into a single command?

There are several ways to tackle this issue:

create a custom shell script that does a double commit invocation
create a git alias that does a double commit invocation
use a native shell solution

I am hesitant to pollute my dotfiles with more scripts, hence I’d rather find a native shell solution.

I came up with this very simple idea: since I use zsh, leverage the repeat built-in:

repeat word do list done

word is expanded and treated as an arithmetic expression, which must evaluate to a number n. list is then executed n times.

% repeat 2 git commit -a -m "feat: bootstrap time travel"

This only works iff I really want to commit everything that is in the staging area (the git index).

— § —

Reply via email

#dev #pre-commit

★ Using keep-sorted to keep your ~/.gitconfig tidy

serendipity@perrotta.dev (Thiago Perrotta) — Thu, 06 Mar 2025 22:53:23 +0100

♠ I tend to sort my ~/.gitconfig headings to keep the config tidy.

An example:

[...]
# https://git-scm.com/docs/git-rerere
[rerere]
	autoUpdate = true
	enabled = true

[status]
	# Show individual files in untracked directories.
	showUntrackedFiles = all
	short = true
	branch = true

[submodule]
	# Clone new submodules in parallel with as many jobs.
	fetchJobs = 0
[...]

This was done manually. Until…today.

It has just occurred to me¹ I could use keep-sorted with the headings.

Previously:

— keep-sorted — pre-commit

But…how?!

The usage to sort, say, aliases, is quite straightforward²:

[alias]
	# keep-sorted begin
	bd = !branch="$(git branch --show-current)" && git default && git branch -D "${branch:-$1}"
	blank = desc -m \"blank commit\"
	cm = commit
	co = checkout
	cp = cherry-pick
	dc = diff --cached
	default = !git checkout main &>/dev/null || git checkout master
	desc = commit --allow-empty -n
	emerge = !git add -A . && git amend -n && git pushm --force-with-lease
	nb = switch --create
        [...]
	# keep-sorted finish

How can you apply the same technique to headings?

The out-of-the-box utilization is OK-ish, but not great:

From:

# keep-sorted begin
[fetch]
	prune = true

[help]
	autocorrect = 10

[format]
	# Automatically sign-off patches when using format-patch.
	signoff = true
# keep-sorted finish

To:

# keep-sorted start

[fetch]
	prune = true
[format]
	# Automatically sign-off patches when using format-patch.
	signoff = true
[help]
	autocorrect = 10
# keep-sorted end

(including the blank line at the top)

It turns out all we have to do is a little bit of tweaking:

# keep-sorted begin block=yes newline_separated=yes
[fetch]
	prune = true

[help]
	autocorrect = 10

[format]
	# Automatically sign-off patches when using format-patch.
	signoff = true
# keep-sorted finish

…which then becomes:

# keep-sorted begin block=yes newline_separated=yes
[fetch]
	prune = true

[format]
	# Automatically sign-off patches when using format-patch.
	signoff = true

[help]
	autocorrect = 10
# keep-sorted finish

How nice!

Can you even imagine? Having original ideas? In 2025? Without LLMs?! Is this even real?? ↩︎
I replaced start -> begin and end -> finish otherwise keep-sorted would sort this blog post itself. If you copy and paste this example, remember to change the words back to the original ones. ↩︎

— § —

Reply via email

#bestof #dev #git #pre-commit

★ GitHub Distributed CODEOWNERS

serendipity@perrotta.dev (Thiago Perrotta) — Wed, 12 Feb 2025 13:57:29 +0100

♠ We recently adopted Andrew Ring’s “Distributed CODEOWNERS” (github) in some parts of our codebase.

You know the concept of OWNERS if you’ve worked at Google, or in the Chromium codebase. As far as I know, they were the pioneers of this concept.

Outside Google, GitHub has popularized it with CODEOWNERS. It’s pretty much the same concept, it’s just implemented differently.

Google #

You have distributed OWNERS files throughout the whole repository. For example:

java/
  OWNERS (a)
  project1/
    OWNERS (b)
  project2/
    OWNERS (c)

OWNERS in (b) have CL (ChangeList) / PR (Pull Request) approval permissions for java/project1, but not for java/project2.

OWNERS in (a) have approval permissions for both projects.

GitHub #

There’s a single .github/CODEOWNERS file that controls the whole repository:

* a
/java/project1 b
/java/project2 c

Distributed CODEOWNERS #

What if we wanted to adopt the Google approach in GitHub? Use Andrew’s tool!

What does it do? It collects all scattered OWNERS in the repository and consolidates them in a single .github/CODEOWNERS file, expected by GitHub. Win-win!

I like to adopt it with a pre-commit hook:

repos:
  - repo: https://github.com/andrewring/github-distributed-owners
    rev: v0.1.9
    hooks:
      - id: github-distributed-owners
        files: .*/OWNERS$|^\.github/CODEOWNERS$

Then it gets automatically managed via CI and/or local commits. It can also be triggered manually with:

pre-commit run -a github-distributed-codeowners

— § —

Reply via email

#bestof #dev #git #pre-commit

codespell: ignore words

serendipity@perrotta.dev (Thiago Perrotta) — Mon, 20 Jan 2025 10:54:55 -0300

♠ When using codespell to spell check a git repository, sometimes you may run into false positive words.

There are two ways to whitelist them.

The first one is via command-line arguments:

codespell -L=ist -L=regio

codespell -L=ist,regio

The second one is with a dedicated .ignore-words.txt file:

% cat .ignore-words.txt
# keep-sorted start
ist
regio
# keep-sorted end
% codespell -I .ignore-words.txt

— § —

Reply via email

#dev #pre-commit

★ YAML: enforce schema linting

serendipity@perrotta.dev (Thiago Perrotta) — Thu, 09 Jan 2025 13:22:32 -0300

♠ YAML files can have a schema associated to them.

Schema store is a popular source for schemas. Or you could write your own.

JSON Schemas work on YAML files just fine.

Let’s say you found a schema. For example, for JSON Patch.

Assume this sample .yaml file:

- op: remove
  path: /spec/syncPolicy/automated

How to enforce the schema above in CI?

One approach I like is with YAML Language Server by Red Hat.

You start by annotating the file with the desired schema:

# yaml-language-server: $schema=https://json.schemastore.org/json-patch.json
- op: remove
  path: /spec/syncPolicy/automated

And then you integrate it with a CI tool that is aware of it. I like pre-commit.com. Assuming you follow the pre-commit setup, integrate the hook:

repos:
  - repo: https://github.com/jmlrt/check-yamlschema
    rev: v0.0.4
    hooks:
      - id: check-yamlschema

Then pre-commit run [--all-files] check-yamlschema shall perform its job.

https://github.com/jmlrt/check-yamlschema:

A CLI and pre-commit hook for jsonschema validation in YAML files with multiple documents

Parse multi-documents YAML files, look for inline schema comments, and validate the documents according to their schema.

The ‘any’ schema¹ can be handy as an opt-out mechanism:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://json.schemastore.org/any.json",
  "anyOf": [
    { "type": "object" },
    { "type": "array" },
    { "type": "string" },
    { "type": "number" },
    { "type": "boolean" },
    { "type": "null" }
  ]
}

Submitted by yours truly. ↩︎

— § —

Reply via email

#bestof #dev #pre-commit

pre-commit + vim integration

serendipity@perrotta.dev (Thiago Perrotta) — Sun, 05 Jan 2025 01:46:06 -0300

♠ Is it possible to transparently invoke pre-commit in the background upon saving in vim (:w)?

Initially I found this Stack Overflow question, but it had no answer.

Digging a bit deeper I realized that I could do it with an autocmd:

autocmd BufWritePost * execute '! if [ -e $(git root)/.pre-commit-config.yaml >/dev/null 2>&1 ]; then pre-commit run; fi'

That trashes the screen upon completion. This can be resolved by invoking :redraw! afterwards:

autocmd BufWritePost * execute '! if [ -e $(git root)/.pre-commit-config.yaml >/dev/null 2>&1 ]; then pre-commit run; fi' | redraw!

We can be less intrusive by prepending silent! so that, even if the command fails, it doesn’t get in the way:

autocmd BufWritePost * silent! execute '! if [ -e $(git root)/.pre-commit-config.yaml >/dev/null 2>&1 ]; then pre-commit run; fi' | redraw!

And last but not least, ideally pre-commit should run only on the file associated with the current buffer. Pass --files to pre-commit with the % argument:

autocmd BufWritePost * silent! execute '! if [ -e $(git root)/.pre-commit-config.yaml >/dev/null 2>&1 ]; then pre-commit run --files %; fi' | redraw!

This approach works as intended. There’s a brief period in which you can see pre-commit running in the background. It’s arguable whether that’s a feature or a bug.

Ah, and git root is simply an alias in my ~/.gitconfig:

root = rev-parse --show-toplevel

Whenever there are pre-commit violations (e.g. end-of-file-fixer) they are applied inplace. There’s no need to run :e!.

This is bare, but I am happy.

Perhaps the only missing piece is to extend it in such a way it’s only executed in the repositories I opt into.

If this turns out to be too intrusive, I am thinking to switch to using a vim keymap e.g. <leader>p instead of an autocmd.

— § —

Reply via email

#dev #pre-commit #vim

Pin all github actions

serendipity@perrotta.dev (Thiago Perrotta) — Sat, 28 Dec 2024 20:49:27 -0300

♠ via Connor Edwards:

Since git tags can be moved arbitrarily by the author, there’s nothing stopping them going rogue and changing the Action to be malicious at any time. The author could change the code to exfiltrate the repository itself, the GITHUB_TOKEN (which in many cases can have more power than it should), or other CI/CD secrets.

Hence, bad:

uses: actions/checkout@v4

Good:

uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4

I find an effective way to address and enforce this practice is via pre-commit:

- repo: https://github.com/lalten/check-gha-pinning
  rev: v1.2.0 # or whatever is the latest version
  hooks:
    - id: check-gha-pinning

Try this to check whether there are any existing violations in your repositories:

% pre-commit try-repo https://github.com/lalten/check-gha-pinning --all-files

I had some in my blog:

GitHub Actions Pinning...................................................Failed
- hook id: check-gha-pinning
- exit code: 1

.github/workflows/gh-pages.yml:22: peaceiris/actions-hugo@v3 is not pinned to commit (should be e3b661c523413d13d642651a5ba5fc0d2b344c0d # v3)
.github/workflows/gh-pages.yml:27: extractions/setup-just@v2 is not pinned to commit (should be dd310ad5a97d8e7b41793f8ef055398d51ad4de6 # v2.0.0)
.github/workflows/gh-pages.yml:33: peaceiris/actions-gh-pages@v4 is not pinned to commit (should be e9c66a37f080288a11235e32cbe2dc5fb3a679cc # v4)

They are now fixed.

— § —

Reply via email

#dev #pre-commit #security

Optimize images

serendipity@perrotta.dev (Thiago Perrotta) — Sat, 28 Dec 2024 16:14:19 -0300

♠ I like to use oxipng to optimize PNG (and other) images.

There’s a pre-commit hook available, therefore enforcing it is a breeze:

- repo: https://github.com/shssoichiro/oxipng
  rev: v9.1.3
  hooks:
    - id: oxipng

Surely you could run it in github actions as well, if you don’t use pre-commit.com.

If I need to use it as an one-off, I can either run the oxipng binary directly, or use a web version such as Squoosh (via Simon Willison).

— § —

Reply via email

#dev #pre-commit

★ Keep sorted

serendipity@perrotta.dev (Thiago Perrotta) — Thu, 26 Dec 2024 15:07:02 -0300

♠ keep-sorted is, by far, one of my favorite tools to enforce tidying up a codebase.

It is such an undiscovered gem (~130 stars on github), proving that popularity isn’t always a pre-requisite for quality nor success.

It started as someone’s 20% project at Google. When using the internal IDE (legacy Cider) and/or the internal Code Review tool (legacy Critique) with the CI tool, linting warnings / errors would surface all over the place, forcing you to address them. It was also possible to have them automatically fixed.

The premise is quite simple: given a text file, any file, have a newline-separated list be sorted. For example, this hypothetical Java array:

String[] capitals = {
    "Tokyo",
    "Paris",
    "London",
    "Berlin",
    "New Delhi"
};

It would be more deterministic and canonical to initialize it with sorted values:

String[] capitals = {
    "Berlin",
    "London",
    "New Delhi"
    "Paris",
    "Tokyo",
};

:sort in vim is my favorite way to do so.

By doing this, you make it easier for the next developer to add a new item:

String[] capitals = {
    "Berlin",
    "London",
    "New Delhi"
    "Paris",
    "Rio de Janeiro",
    "Tokyo",
};

But surely at some point one coaster¹ will ruin it:

String[] capitals = {
    "Berlin",
    "London",
    "New Delhi"
    "Paris",
    "Rio de Janeiro",
    "Tokyo",
    "Ottawa",
};

I mean, it’s only natural to append new items to the end, right? Who said it has to be sorted?

Perhaps you could attempt to address this by adding a comment at the top:

String[] capitals = {
    // keep this list sorted, as this is the order displayed to the end-user in a dropdown menu
    "Berlin",
    "London",
    "New Delhi"
    "Paris",
    "Rio de Janeiro",
    "Tokyo",
};

But developers don’t read documentation, right?² Some of them do, but they don’t have time to read documentation all day. There are so many tickets in our JIRA backlog!

Surely you could force them to read the documentation by the means of a mandatory process, but it adds overhead and it’s costly.

Maybe you will make a comment to address it during code review. But who guarantees the review will be always sent to you? Also, it probably shouldn’t.

Then why not fix it in code? Let the programming language enforce the invariant for you.

import java.util.Arrays;

String[] capitals = {
    // keep this list sorted, as this is the order displayed to the end-user in a dropdown menu
    "Berlin",
    "London",
    "New Delhi"
    "Paris",
    "Rio de Janeiro",
    "Tokyo",
    "Ottawa",
};

Arrays.sort(capitals); // sort in place

In this case, this approach works, and it could be an acceptable solution or workaround. But it won’t always work. What if we had an YAML file instead?

capitals:
  # keep this list sorted, as this is the order displayed to the end-user in a dropdown menu
  - Berlin
  - London
  - New Delhi
  - Paris
  - Rio de Janeiro
  - Tokyo
  - Ottawa

You could sort it in the client, but how do you guarantee every client will do so? Maybe you don’t control all clients.

Now, remember the title of this post? If you integrate keep-sorted in this file³:

capitals:
  # keep-sortedd start
  - Berlin
  - London
  - New Delhi
  - Paris
  - Rio de Janeiro
  - Tokyo
  - Ottawa
  # keep-sortedd end

…the end result will be:

capitals:
  # keep-sorted start
  - Berlin
  - London
  - New Delhi
  - Ottawa
  - Paris
  - Rio de Janeiro
  - Tokyo
  # keep-sorted end

This invariant can be enforced at CI time. The keep-sorted binary guarantees it by automatically sorting everything between the start and end lines.

It’s possible to fine tune it a bit, but there’s no need to in 99% of the cases. Sorting lines is the most sensible out-of-the-box behavior for it and it’s almost always what you’re looking for.

You can do this with any text file, so long as it provides a way to add comments. This means JSON is out. YAML, protocol buffers, programming languages, markdown…almost everything else works. JSON is the ugly duck in this context.

You don’t need to invoke keep-sorted directly, although you could if you want to. Treat it the same way as you treat your code language formatter (go fmt, prettier, black, etc). Have it be automatically invoked by your IDE, or by your pre-commit framework, or by CI (e.g. github actions).

You don’t even need to teach people about it. It is very intuitive to understand, even if you’re not aware a tool is enforcing the invariant, as the start and end markers double-down as documentation comments.

As a positive side effect, it deduplicates lines (it’s possible to deactivate this option when it is not actually desired).

I peer-bonused the creator of this tool. It’s in my opinion one of the simplest (yet effective!) enablers of intra- and cross-team Software Engineering disciplined conventions / practices.

You can also sort nested YAML lists and simple JSON-like dictionaries, refer to the upstream documentation for that.

Not necessarily a coaster: believers and grinders can also make accidental mistakes, we humans are all flawed, that’s why we have CI, eh? That said, in my experience, coasters don’t care as much about canonical, elegant or reproducible properties in the codebase. ↩︎
Documentation is only intended to please QA, right?!1! ↩︎
The typo is intentional. Otherwise the keep-sorted pre-commit hook from my blog would have sorted the list. Unless I added an exclusion rule for it, which seems unnecessary. Minimizing exclusions is better. ↩︎

— § —

Reply via email

#bestof #dev #pre-commit

★ Pre-commit

serendipity@perrotta.dev (Thiago Perrotta) — Sat, 21 Dec 2024 23:24:23 -0300

♠ pre-commit is a CI framework for git. For those who are used to google3 tooling: it’s akin to the configuration part of TAP Presubmit.

You can plug in linters, formatters, code analyzers…pretty much any tool or binary that analyzes files, potentially emitting errors whenever style or policy violations occur, is a potential good fit for a pre-commit hook.

Once properly configured the whole pipeline can be run locally with pre-commit run. By default, only the stashed files are inspected. In order to consider every file in the repository, pass --all-files. Modified files that were not yet git added are not included.

When configuring CI, the same command is used in cloud pipelines: it’s a single configuration for both local and CI runs.

All the configuration lives in a single YAML file, .pre-commit-config.yaml, in the repository root.

Because it is written in Python, the whole ecosystem around the language is very well supported, first-class. That said, many other languages are also supported. For the ones that are not, there is a escape hatch – the “system” language.

Here is an example .pre-commit-config.yaml file – the one for this blog:

repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v5.0.0
    hooks:
      # keep-sorted start
      - id: check-executables-have-shebangs
      - id: check-symlinks
      - id: check-yaml
      - id: detect-private-key
      - id: pretty-format-json
        args:
          - --autofix
          - --no-sort-keys
      - id: sort-simple-yaml
      - id: trailing-whitespace
        # keep-sorted end
  - repo: https://github.com/google/keep-sorted
    rev: v0.5.1
    hooks:
      - id: keep-sorted
  - repo: https://github.com/google/yamlfmt
    rev: v0.14.0
    hooks:
      - id: yamlfmt
  - repo: https://github.com/codespell-project/codespell
    rev: v2.3.0
    hooks:
      - id: codespell
        exclude: ^content/posts/(2014|2015|2016)|content/posts/2024-07-09-kubectl-get-secret-with-jsonpath-add-newline.md|content/posts/2024-06-27-a-little.md
        args:
          # keep-sorted start
          - -L=DeVault
          - -L=als
          - -L=ist
          - -L=sive
          # keep-sorted end
  - repo: https://github.com/jmlrt/check-yamlschema
    rev: v0.0.4
    hooks:
      - id: check-yamlschema

Repository versions are always pinned. You would think this is tedious, but it’s highly desirable for reproducibility, and they can all be automatically updated with pre-commit autoupdate.

To run only a single hook: pre-commit run [--all-files] codespell.

How do you install pre-commit in the first place? Use pip, or brew (macOS), or your favorite package manager.

Hooks are configured in .pre-commit-hooks.yaml files in their respective repositories. The codespell configuration is an example, from codespell:

- id: codespell
  name: codespell
  description: Checks for common misspellings in text files.
  entry: codespell
  language: python
  types: [text]

…it’s just metadata that teaches the framework to run the codespell python file, and that it should only run in text files.

The definition of what a text file is lives in the identify pre-commit library, this file has all the mappings. You could also specify shell, for example.

What if an extension isn’t supported? Instead of using types:, use file: with a regex pattern to match, such as \.sh$.

What if an upstream tool does not have a .pre-commit-hooks.yaml file? I wrote a post with a workaround wherein you can define your own.

How do you try out new hooks? One way is to use pre-commit try-repo. Another way is to add them to your config and then run each hook individually.

Are there similar tools? There’s husky, but it’s too nodejs / web centric. I like pre-commit.com better.

Any git repository that is owned my multiple people and need to follow well-defined practices and conventions should, in general, adopt a pre-commit framework. pre-commit.com is one possible solution that is great at its job.

References #

https://whynothugo.nl/journal/2023/01/12/notes-on-pre-commit/

— § —

Reply via email

#bestof #dev #pre-commit

★ pre-commit: create hooks for unsupported tools

serendipity@perrotta.dev (Thiago Perrotta) — Tue, 17 Dec 2024 22:46:41 -0300

♠ When using pre-commit.com, in an ideal world, every formatter / linter / code analyzer would have a .pre-commit-config.yaml file in its repository root.

In the real world, that’s not always the case.

A recent example: cloudflare/pint:

Prometheus rule linter/validator

It is a golang binary that lints prometheus rules.

Can we bridge the gap?

The end goal is the ability to run pre-commit run --all-files pint in our git repository.

For that, we’ll need to define a local / custom hook in our ~/.pre-commit-config.yaml:

repos:
  - repo: local
    hooks:
      - id: pint
        name: Validate prometheus rules with pint
        language: golang
        additional_dependencies:
          - github.com/cloudflare/pint/cmd/pint@v0.69.1
        entry: pint --offline lint
        types:
          - yaml
        files: ^helm/clustermon-alerts/rules/

That simply works™. How did I get there? Starting with the references:

The key part is language: golang + additional_dependencies.

Specifying additional_dependencies is a bit tricky. Initially, I did:

github.com/cloudflare/pint@v0.69.1

…but that yields nothing. In fact:

% go run github.com/cloudflare/pint@v0.69.1
go: github.com/cloudflare/pint@v0.69.1: module github.com/cloudflare/pint@v0.69.1 found, but does not contain package github.com/cloudflare/pint

We must always specify the module that contains the main function with go run, and that is cmd/pint.

— § —

Reply via email

#bestof #dev #pre-commit

Finding linters

serendipity@perrotta.dev (Thiago Perrotta) — Sun, 08 Dec 2024 23:40:26 +0100

♠ A no-brainer and effective way to increase code quality in an organization is by the means of incorporating linters into your CI/CD pipeline.

But where can you find them?

Some sources of inspiration include curated “super-mega-hyper” linter packages, such as:

I am not generally a fan of employing these collections directly because it’s not possible to exert tight control over them, and there’s no security or reproducibility guarantee of their pipeline.

Instead, I cherry-pick interesting / useful linters from their packs, effectively using them as serendipity sources for linters.

Bonus points whenever there’s out-of-the-box integration with the pre-commit.com framework i.e. whenever there’s a .pre-commit-config.yaml file present at the root of the git repo.

— § —

Reply via email

#dev #pre-commit

Rename files in bulk (cont)

serendipity@perrotta.dev (Thiago Perrotta) — Tue, 26 Nov 2024 11:50:06 +0100

♠ Deeper into the rabbit hole:

% fd Dockerfile.dockerignore | xargs -n 1 rename 's/Dockerfile\.dockerignore/.dockerignore/'

The pre-commit identify library currently mistags these files as Dockerfiles, even though they are not¹, which creates all sorts of issues.

They are akin to .gitignore files in terms of structure. ↩︎

— § —

Reply via email

#dev #pre-commit

pre-commit: additional dependencies in golang

serendipity@perrotta.dev (Thiago Perrotta) — Thu, 21 Nov 2024 12:36:35 +0100

♠ When working with pre-commit.com and specifying language: golang for a given hook, you may want to install dependencies as part of the hook bootstrapping process.

Recently I needed to do so for yq¹:

The .pre-commit-config.yaml looked like this:

repos:
  - repo: local
    hooks:
      - id: helm-dirname-must-match-chart-name
        name: Helm chart directory name must match the chart name
        files: /Chart\.(yml|yaml)$
        entry: ci/helm_check_match_dirname_chart_name.sh
        language: golang
        additional_dependencies:
          - https://github.com/mikefarah/yq

Context for the hook and the script: https://stackoverflow.com/q/79166730/1745064.

It didn’t work. It’s necessary to drop the https:// prefix.

repos:
  - repo: local
    hooks:
      - id: helm-dirname-must-match-chart-name
        name: Helm chart directory name must match the chart name
        files: /Chart\.(yml|yaml)$
        entry: ci/helm_check_match_dirname_chart_name.sh
        language: golang
        additional_dependencies:
          - github.com/mikefarah/yq

That didn’t work either. Then I realized I needed to specify an exact version²:

repos:
  - repo: local
    hooks:
      - id: helm-dirname-must-match-chart-name
        name: Helm chart directory name must match the chart name
        files: /Chart\.(yml|yaml)$
        entry: ci/helm_check_match_dirname_chart_name.sh
        language: golang
        additional_dependencies:
          - github.com/mikefarah/yq@v4.44.3

It also didn’t work! There was an error message about the need to specify /v4 in the path for whatever reason:

repos:
  - repo: local
    hooks:
      - id: helm-dirname-must-match-chart-name
        name: Helm chart directory name must match the chart name
        files: /Chart\.(yml|yaml)$
        entry: ci/helm_check_match_dirname_chart_name.sh
        language: golang
        additional_dependencies:
          - github.com/mikefarah/yq/v4@v4.44.3

That worked! Test it:

pre-commit run --all-files helm-dirname-must-match-chart-name [--verbose]

yq is like jq for YAML instead of JSON. ↩︎
@latest would also work. However, for the sake of reproducibility, pinning is more reliable. ↩︎

— § —

Reply via email

#dev #pre-commit

Pre-Commit on ¬ just serendipity 🍀

pre-commit: authentication in internal github repos

★ Worktrunk

Setup #

Workflow #

migrating from pre-commit to prek

pre-commit: periodic garbage collection

github workflow: external script

pre-commit: hazmat n1

npm error code EBADF

new pre-commit hook: check-bash-shebang

go/keep-sorted

pre-commit: autoupdate

"Fix typo"

★ check-json-schema: pre-commit hook

Helm repository with a trailing slash

pre-commit: add hugo build

pre-commit: make a hook opt-in

pre-commit: pin dependencies with --freeze

★ git freeze, git thaw

git-freeze #

git-thaw #

Post-AI cleanup #

Usage #

YOLO: debug only after push

keep-sorted, with quotes

commit, pre-commit, commit

★ Using keep-sorted to keep your ~/.gitconfig tidy

★ GitHub Distributed CODEOWNERS

Google #

GitHub #

Distributed CODEOWNERS #

codespell: ignore words

★ YAML: enforce schema linting

pre-commit + vim integration

Pin all github actions

Optimize images

★ Keep sorted

★ Pre-commit

References #

★ pre-commit: create hooks for unsupported tools

Finding linters

Rename files in bulk (cont)

pre-commit: additional dependencies in golang

`git-freeze` #

`git-thaw` #