I built our reporting tool myself, with an AI coding agent
In short
- Problem: no reliable view of how engineering time and cost were distributed across teams, projects and epics. KPI reporting was assembled by hand and out of date on arrival.
- Decision: build a small internal application rather than buy a platform — specification, KPI definitions and data model mine, implementation with a coding agent.
- Result: weekly snapshots generated automatically, a KPI dashboard across six roles, and reporting that takes minutes instead of a day.
- Timeframe: shipped alongside the day job, not instead of it.
Context
A Tech and Product department of twenty-five, working in Jira, across a portfolio of live products. Management needed to know where engineering effort was going — by team, by project, by epic — and what it cost.
The problem
I could not answer “how much of last quarter went into this product” in less than half a day of spreadsheet work. For a department head, that is not an inconvenience; it is a gap in the job.
The KPI reporting was worse. It existed, but it was assembled manually, which meant three things followed automatically. It was late — by the time a number was compiled it described a situation that had already changed. It was inconsistent — whoever assembled it made small judgement calls, and those calls were not written down anywhere. And it was fragile, in the specific way that anything living in one person’s spreadsheet is fragile.
There is also a subtler cost. When producing a number is expensive, you stop asking questions that would need one. You navigate on intuition and call it experience.
Constraints
No budget for a platform, and I would not have wanted one — the questions I needed answered were specific to how this department works, and a generic tool would have answered adjacent questions instead.
No spare engineer to assign. Pulling someone off product work to build internal tooling would have been the wrong trade, and I would have had to defend it.
The data lived in two places with different shapes: Jira for worklogs and issue structure, and our team metrics provider for engineering activity indexes. And whatever got built had to be maintainable by someone other than me, because internal tooling that only its author can run is a liability disguised as an asset.
What I did
Wrote the specification before anything else, including what not to measure. This was the part that took the longest and mattered most. The application produces time and cost allocation per developer, project and epic, and a KPI dashboard covering twenty-eight indicators across six roles.
What it deliberately does not do is rank individuals. Every KPI definition went into an in-app guide explaining what the indicator measures and what it does not, because a number without its definition attached will eventually be used for something its author never intended. That guide is part of the product, not documentation about it.
Chose historical snapshots over live queries. A dashboard that always shows the current state cannot answer “was this better last quarter”, which is the only question that makes a metric useful. The application stores a snapshot on a schedule and charts the trend. It also means the report a director saw last Friday still exists, unchanged, when they ask about it on Monday.
Made all configuration data, not files. Teams to Jira boards, members and roles, per-KPI settings, API credentials — all in the database, credentials encrypted. This was the maintainability requirement made concrete: someone else can change how a team maps to a board without touching a deploy.
Automated the delivery of the report, not just its generation. A scheduler container builds the weekly snapshot and pushes it to a spreadsheet where non-technical stakeholders already live. A report that requires someone to remember to run it is a report that stops existing in month three.
Then built it, hands-on, with a coding agent. Laravel, Dockerised, MySQL for sessions and cache. I wrote the spec, the KPI definitions and the data model; the agent did the implementation work under review.
Two things about that, stated plainly. First, it changed the economics: this was a project I would otherwise have had to either not do or hand to someone whose time was worth more elsewhere. Second, it required more discipline, not less. Credential handling got reviewed line by line. Secrets that had been committed in an earlier iteration of this work got rotated, with the rotation documented. The agent is fast and confident, and confidence is exactly the failure mode you have to design around.
What did not work: several of the KPIs in my first draft turned out to be uninformative — they moved with team size or sprint length rather than with anything I cared about. I removed them. A metric that has to be explained every time it is presented is not a metric, it is a footnote.
Result
The department reports from the application. Time and cost allocation is available on demand, the weekly snapshot arrives without anyone touching it, and trends exist because history is stored rather than overwritten.
The change I did not anticipate is in my own behaviour. Because a number now costs nothing to produce, I ask more questions — including ones where the answer contradicts what I assumed, which happens more often than is comfortable.
What I would change
I would build the KPI guide first, not alongside. Writing down what an indicator does not mean turned out to be the strongest filter for whether it deserved to exist at all — I would have shipped fewer, better indicators if I had applied that filter before implementing them.
And I would have set up the secret rotation process at the start rather than in response to finding a problem. That is not a lesson about AI. That is a lesson I already knew.