We built 3 new research tools in one session. Here's what broke — and why that's the point.
Building three research tools for Envest surfaced six real bugs. Two of them passed every test and only showed up on screen.

A recent analysis in Envest — P/E against real sector history, Elliott Wave structure, correlation with a real macro series, all three on the same company, same day — needed three capabilities that didn't exist in the product an hour before we started. This is what it actually took to build them, and the six real bugs that showed up doing it, because "it compiled and the tests passed" and "it actually works" turned out to be two different claims, twice.
What got built
- A new sector-valuation tool — a company's live P/E against its sector peers' median, plus where that sits against the sector's own ~10-year historical range (via a third-party market-data provider's own multiples history, not bounded by our local ingestion window). Correctly told apart from two related, similar-sounding questions in every test case on the first pass.
- A real methodology bug, found building it, not after publishing: the first version averaged peer P/E. One company in this exact peer set had near-zero GAAP earnings for years — real, not a data error — producing a genuine annual P/E over 1,000 in one year. Fixed to median in both layers (today's snapshot and the historical band) — the band went from an implausible 27.59–147.29 to a real-looking 20.84–37.91.
- A generic chart-annotation layer — pivot points and support/resistance levels can now be drawn on any Q&A price chart. Built once, generically, so Elliott isn't the last pattern-detection tool that needs this.
- A new Elliott Wave structure tool — deterministic zigzag pivot detection plus a rule-compliance checker against Elliott's own 3 structural rules. Every test case was hand-verified before being trusted, not just run — one real bug in the pivot-detection fallback logic was found and fixed this way, not after the fact.
- A routing gap found and fixed: the system's own top-level query router didn't recognize "Elliott Wave structure" as an analytical question, and sent it to document search instead — where it correctly found nothing and said so. Fixed by extending that router's own scope description; confirmed with a live rerun, not assumed.
- A bug found only by actually recording: a chart with 500 daily points broke the frontend's word-reveal animation (built for prose), and the chart couldn't render until more than 15 seconds had passed. Fixed by reusing the sampling convention every other price chart in the codebase already used — 500 → 151 points, with every real pivot forced into the sample regardless of step, so a wave point can never silently vanish from its own structure chart. Pivot detection still runs on the full daily series; only the display copy is thinned.
- Another one, also found only by recording: the marker-rendering logic from item 3 was built and type-checked, but the running service was still serving a build from a week earlier — the image had been rebuilt, but the container had never been restarted from it. The chart rendered; the wave markers didn't — 0 of 6, confirmed by counting actual DOM elements before and after the fix.
- A new macro-correlation tool — Pearson correlation between the period-over-period change in a real macro series (Fed Funds Rate, Treasury yield, CPI — not the level, which would produce a misleading number) and a company's return over the same window, aligned to the macro series' own dates using the price on or before each one, never after. Every test case on the math was hand-verified, which caught an error in our own test data along the way: a constant series produces an undefined correlation, not "perfect proportionality", which we'd assumed going in.
What "verified" actually meant here
The four example answers in that analysis are real answers from the real pipeline — independently confirmed as grounded, with citations that check out — not assembled from isolated function calls. In the macro-correlation case, the system's own anti-fabrication check flagged an issue once and self-corrected on the single retry. The full automated test suite passed 139 of 141 cases; the two exceptions were pre-existing, already-documented flakiness, confirmed unrelated to this work.
But the bugs in items 6 and 7 are the ones worth sitting with. Both tools passed every unit test, every type check, every clean build, before either bug was found. Neither was visible from the backend at all — one was a frontend animation racing against a payload size nobody had hit before, the other was a service quietly running last week's code while this week's code sat built and unused. Both only showed up once the actual recording started and the actual chart didn't appear on screen.
"It's verified" has at least three meanings: the data is correct, the code is correct, and what the user actually sees is correct.
This session is a real case where the first two were true and the third wasn't, twice, for two unrelated reasons.
This work shipped inside Envest, our research and portfolio intelligence platform for equity analysts.
Share this insight