When we developed the LLM-powered AI assistant part of our product offering we implemented a direct feedback mechanism with thumbs up/down and an input field. We thought that channel was enough to know when things go wrong and then improve the assistant using that feedback.But the reality was different — it was 100 times harder to incorporate the collected feedback from the AI assistant into the improvement pipeline.Also, our users do not like to give us feedback — less than 10% of users are giving feedback so it’s biased.We added Heap as well - it only provides the number of clicks inside and outside of the AI assistant.
