dhruv's devlog

#002: Can GitHub Copilot agent help with go toolchain upgrades?


Go 1.25.0 dropped as a stable release a few days ago. Upgrading the toolchain for my Go tools has always been refreshingly straightforward (one of the language’s best qualities), but doing it across 10+ tools gets old real fast.

Since I was staring down yet another round of mechanical upgrades, I figured this was a good opportunity to try something different — let GitHub Copilot’s coding agent handle the grunt work. After manually upgrading a couple of projects to get a feel for the process, I fired up several Copilot agent jobs with this prompt:

Upgrade this project to use go 1.25.0. You'll need to do the following:

- Upgrade version in go.mod
- Update the version specified in the workflows defined in `.github`
- Migrate to use golangci-lint `2.4.0`
- Ensure build, lint checks, and tests pass. If lint rule `var-naming` fails,
    remove it from the config

The explicit instructions are to reduce the amount of back and forths needed in this this asychronous mode of operation. Otherwise, it’s just easier to do this using a local agent.

The results were mixed, but mostly encouraging. Most agents finished in 7-8 minutes (not lightning fast, but not terrible either), handling the straightforward cases with ease. Where they all tripped up was on the var-naming lint rule from revive. In repositories where this rule was explicitly enabled, the agents correctly removed it. But two repositories had this rule enabled through golangci-lint’s default configuration, which required an explicit exclusion in the config file — a nuance that escaped every agent.

I didn’t want to begin a back-and-forth with the agents through PR comments, so I fixed those cases manually. I also wanted to see how long it would take me to upgrade the toolchain in the complicated case (to have a reference against the agent), so I recorded it (attached below) — it took me 3.5 minutes.

I’ll be on the lookout for improvements to Copilot’s coding agent. I like the idea of offloading these mechanical tasks to asynchronous agents, especially after I’ve done them once or twice. I will also experiment with other asynchronous agents and models (as of now GitHub doesn’t let you choose the model for its agent).

Hopefully, in the near future, I’ll be able to script all of this up (via official APIs) and fire off asynchronous tasks for toolchain upgrades with a single command.

Wouldn’t that be neat?