A Team Memory System So ML Experiments Stop Repeating Themselves
A structured, searchable registry that captures ML experiment learnings, successes and failures alike, so Claude Code can warn teams before they redo work.
ML research teams rediscover the same insights over and over. Someone runs an experiment, learns that a particular hyperparameter setting does not work, and six weeks later a different person on the team runs almost exactly the same experiment because that knowledge lived in a Slack thread, a notebook, or one researcher's head. research-skills is a prototype built to stop that leak by turning experimental learnings into structured, searchable records that Claude Code can query during future work.
Capturing failures, not just wins
The framing that makes this useful is that it captures failures as deliberately as successes. Most documentation systems only record what worked. But in research, knowing what does not work is often the more valuable signal, because it is the thing people waste the most time rediscovering. The registry is built to prevent teams from repeating failed experiments, forgetting which hyperparameters worked, losing tribal knowledge when a researcher leaves, and re-solving problems that already have known answers.
The design takes cues from Sionic AI's published approach to running very high volumes of ML experiments daily, where institutional memory is not a nicety but a requirement for the pace to stay sane.
Two commands that bracket an experiment
The workflow is intentionally small, built around two commands defined in a CLAUDE.md that Claude Code reads. Before an experiment, a researcher runs /advise and gets warnings about known failures relevant to what they are about to try. After the experiment, they run /retrospective and Claude helps document what happened into a new skill.
That bracket is the whole loop. /advise at the front loads the team's accumulated memory into the moment of highest leverage, right before you commit time and compute. /retrospective at the back closes the loop so the experiment that just finished becomes advice for the next person. The knowledge each skill holds lives in a plugins/ directory of structured documents, with templates/ defining the shape of a new one and scripts/ providing Python utilities for validation and marketplace generation. GitHub Actions validate skill structure on every pull request, so the registry cannot silently fill up with malformed entries.
The takeaway
Institutional knowledge does not get lost because people are careless. It gets lost because capturing it has no natural home in the workflow, so it never happens. The move that makes this work is putting capture and retrieval at the two moments a researcher is already stopping to think, right before an experiment and right after. Structure and CI validation keep the records queryable rather than turning into another dead wiki. Get the timing right and team memory becomes a byproduct of doing the work instead of a chore layered on top of it.
Have something that needs building, or stabilizing?
These notes are the work log. The paid work runs through Moonshine Labs, my product and engineering studio. Tell us what you're building, or grab a call.
Work with us →