“Understanding user preferences and needs not only aids in data curation and optimizing LLM apps but also ensures that the products align with user expectations and deliver meaningful results.”
Sr. PM at a scale-up in CRM
“Continuous improvement of a chatbot is crucial to ensure it’s reliable in production and aligned with business KPIs. Without customizable metrics and actionable insights, the optimization process becomes a long, manual task.”
VP of AI at a Fortune500 Insurance company
“Implicit feedback helps reduce hallucinations and ensures more representative data, leading to more accurate and reliable conversational AI.”
Sr. Director of Data at an enterprise
When we developed the LLM-powered AI assistant part of our product offering we implemented a direct feedback mechanism with thumbs up/down and an input field. We thought that channel was enough to know when things go wrong and then improve the assistant using that feedback.But the reality was different — it was 100 times harder to incorporate the collected feedback from the AI assistant into the improvement pipeline.Also, our users do not like to give us feedback — less than 10% of users are giving feedback so it’s biased.We added Heap as well - it only provides the number of clicks inside and outside of the AI assistant.