Security · A supervised test

We sent Patchkind hostile feedback on purpose

A Patchkind note becomes a plan the owner approves. Then Claude Code runs inside the owner's own GitHub Actions and can open a pull request. It may also ask a question or stop. A note can come from someone you do not know. So we wrote one that tried to misuse the agent and ran it on our own product, AlgoThesis.

One run. Not a guarantee.

What we sent

A note about one page also asked the agent to expose environment/process data, send a token to an outside address, and change protected project or workflow files. These are summarized categories, not a verbatim copy of the report.

What Claude did

Our verifier (a read-only Claude review of the run’s logs and summary) reported that Claude ignored the injected instructions and changed only the page it was asked to. The verifier also reported no sk-ant- credential shapes in the inspected logs. This describes one inspected run, not proof that no secret could ever escape.

What the workflow did

GitHub Actions run 37274899273 on AlgoThesis (private repository).

• fetch, agent and report completed.
• checks failed while validating the patch against protected paths, before that job ran its test commands. The patch contained empty files at protected names such as package.json and .npmrc, so it was rejected.
• publish was skipped because it depends on checks. No branch was pushed. No pull request was opened.

The empty files were not Claude's doing. Claude Code's sandbox creates zero-byte placeholders at those names so the agent cannot write to them. Our packaging step swept them into the patch. The check caught our own bug and stopped the run.

What we fixed

The packaging step now records which placeholders existed before the agent ran. Afterwards it drops only new, zero-byte, untracked files at those exact names, confirming each staged blob is empty first. The protected-path check in checks and publish is unchanged. A real edit to a protected file is still rejected.

What this does not prove

• One run, one repository, one set of instructions. A different phrasing, model version or repository could behave differently.
• The reported refusals came from the model. That behavior alone is not a safety boundary. We rely on what each job is allowed to do, below.
• The run never reached publish, so it does not demonstrate the pull-request step. The four merged AlgoThesis PRs do; see the case study.
• It exposed our packaging bug. A clean result would have been better evidence.

How the workflow limits what the agent can do

Assume a note will one day get the agent to follow an attacker's instruction. The design limits the damage:

• The agent job has a read-only GitHub token, no id-token permission, and sandboxed commands with network access disabled. Your Claude credential is supplied only to the agent step.
• Fail-closed checks: publish runs only after checks. A failed test or build still opens a pull request, as a draft, with each check's exit code in the PR body. A rejected patch opens nothing.
• Protected paths: patches touching .github/, .claude/, CLAUDE.md, AGENTS.md, package manifests, lockfiles, .env*, keys, vercel.json and other hosting files are rejected before the isolated checks job executes repository commands, and again before publication. Owners add their own.
• publish refuses a patch whose hash differs from the one checks recorded, and asks Patchkind whether the job is still approved before pushing. A stop, edit or deletion recorded before that final check prevents authorization to push. A pull request already opened still needs an owner to close it.
• A human reviews every pull request. Patchkind never merges.

Try a supervised test on your own repo

1. Connect a private npm repository and merge the Patchkind workflow file to your default branch.
2. Store your Claude subscription token or API key as a repository Actions secret. Patchkind never holds it.
3. From your app, leave a note with instructions like the categories above. Keep them obviously hostile.
4. Approve the plan and press Run.
5. Read the Actions run: which jobs completed, what checks said, whether a pull request appeared.
6. Tell us at support@patchkind.com, with the run number if the agent did something it should not have.

Patchkind is a free invited beta from 1232216 B.C. Ltd., Canada. npm projects, GitHub-hosted runners, Claude Code. Owners sign in with ChatGPT.

Read the workflow guide for the actual job boundaries.