// HACKER NEWS — CYBERSECURITY
Tracking down a Zsh history data loss bug
For many years, I sometimes discovered that commands I was sure I had run were
no longer present in my Z shell history
file (~/.zsh_history). In this article, I will show you how I tracked down the
bug. Spoiler: ultimately, patching Zsh to make it crash loudly and analyzing the
crash’s core dump was the winning strategy!
Zsh 5.9.2 (released July 12th, 2026) contains a fix for this issue — open it
after reading this investigation to not spoil the fun.
Occasionally, I noticed that commands I knew I executed the day before were
not findable in my shell history, meaning pressing Ctrl+R for backward history
search yielded no results. Whenever I noticed this, my shell history file
contained only very old entries, with years of newer entries missing.
The first few times this happened I just restored my shell history from my daily
backup and did not bother investigating any further. But the issue kept happening.
I noticed that there was no visible corruption in the .zsh_history (no
non-printable characters or incomplete lines of text), and that the number of
lines in the file was not always the same.
What was not clear to me was whether it was Zsh itself, or some other program,
or perhaps the combination of multiple zsh(1)
processes that caused the issue.
I set the following history-related options in my ~/.zshrc:
In practice, this means my shells are separate sessions that all stream their
commands into a shared ~/.zsh_history. The history is intentionally not
shared, so when I want to access entries that another shell wrote, I explicitly
run exec zsh.
When I asked for help on Mastodon in December
2024 (mostly in the hope that
somebody else already encountered and diagnosed this problem), one suggestion I
got was to use file system change monitoring mechanisms like inotify or fsevents
to find the culprit that truncates (or changes?) the Zsh history file.
The next sections walk through the available options on Linux which I tried.