Evaluating AI
Effectiveness
Do what you love and let AI do the rest.
Toby Daniels
Co-Founder, ON_Discourse
Editor’s Note: This is a dispatch Toby wrote from his recent trip to Web Summit in Lisbon.
“When you deploy AI in your business, you have two fundamental paths: you can either enhance the things people love doing or eliminate the things they hate doing. This distinction might sound basic, but it’s crucial.”
This is what Nicholas Durkin, the CTO of Harness, an AI-delivery platform said during a roundtable discussion that Dell and NVIDIA hosted during Web Summit in Lisbon.
Take AI code generation, for example. Developers generally love writing code—it’s their craft and their passion. But when AI tools like code generators were introduced, there was pushback. In fact, recent DORA metrics showed that developers using AI code generation tools were less efficient than they were before adopting them. Why? Because these tools inadvertently disrupted the part of their job they enjoy most—writing code.
Durkin went on to say “It’s like telling a chef, we’ll handle the cooking for you,” but leaving them with all the prep and cleaning instead. Chefs thrive on the act of cooking; they don’t want to lose that joy. Conversely, if you use AI to handle the worst parts of the job—like prep work or cleanup—you empower the chef to focus on what they love. This approach doesn’t just maintain their passion; it enhances their ability to excel.
It’s like telling a chef, we’ll handle the cooking for you, but leaving them with all the prep and cleaning instead.
I spent time with Nick after the roundtable and went much deeper into this topic with him, he went on to outline a model that he uses with his clients. He considers these three metrics when evaluating AI effectiveness:
- Efficiency – Does it make processes faster?
- Reliability – Is the output consistent and of high quality?
- User Experience – Does it make people feel good about their work?
But the most critical factor is alignment with people’s passions. If your AI diminishes the best parts of someone’s job, you’ll face resistance. If it tackles the worst parts, people will embrace it. Focus on “love” and “hate.” Build AI for things people love to do but can’t due to limitations, or for things they can do but don’t want to because it’s boring or repetitive. That’s where AI can make the biggest impact.
We run events every week. If you want to participate, inquire about membership here. If you want to keep up with the perspectives that we hear, you can subscribe to our weekly newsletter.