East West Hospitality

Blake Fleischer

Evidence-Based AI Champion

AI ultimately changes how humans work. Its answers are already shaping decisions today; they arrive quickly, at scale, and often appear polished. Our team has found the effort needed to employ these generated answers effectively begins before a question is asked and continues long after an answer arrives.

The data team at East West uses AI to help explore unfamiliar code, investigate problems, draft changes and run tests. It often gets us to a useful initial result faster. We have found, however, that AI fluency and domain knowledge play different roles. Someone can be skilled at directing a model, but judging its output typically requires understanding the details. AI models do not inherently know a system’s history or what a metric means within an organization, and it is often the edge cases that determine whether an answer is right.

Combining AI Fluency with Domain Expertise

The difference matters most when the output looks finished. A person who knows the domain can narrow the question, recognize a detour, and identify the use cases a solution must handle. Someone new to the domain may use the same tool just as fluently, yet lack the knowledge to challenge a plausible result. Knowing how to use AI does not mean one can determine when the answer is wrong. That is why we continue to develop domain expertise alongside AI fluency: it shapes the questions we ask and the evidence we seek.

 Answers from an AI model, no matter how reassuring, are not independent evidence. 

As an example, consider an explanation for a change in reported revenue. AI today can retrieve a metric definition, compare periods, reconcile numbers, and draft a persuasive account. It may even find a discrepancy. But the explanation depends on which charges and fees belong in the metric and whether the available definition reflects what the organization ultimately decided to measure. A colleague who knows that history can head straight to the consequential question. AI can reconcile every number and still answer the wrong business question.

Turning AI Outputs into Verifiable Insights

AI can also help us enter unfamiliar territory. Our teams have found it helpful to develop and hone written materials that carry domain expertise into our conversational AI tooling. Even with that context, we ask a few questions before accepting an answer: what business records or data would change this conclusion? Which edge case might expose a mistake? What is the original source for this assertion (software codebase, research paper, etc.), and does it support the claim? We then inspect the source ourselves and interpret what the results mean. The check that matters most may be the one AI never suggests. Answers from an AI model, no matter how reassuring, are not independent evidence.

Creating Space for Critical Evaluation

Leaders play an important role in making verification possible. A result can pass every test we run and still rely on an assumption we never examined. On our team, that means making room to question the tests themselves, with the effort matched to the stakes: a meeting idea needs less scrutiny than a change to production systems.

For consequential work, we need access to the underlying records and decisions, people who know the domain well enough to challenge an interpretation, and the authority to pause action when the evidence is insufficient. If speed is the only measure, the necessary work can look like a delay.

Building Trust through Evidence and Accountability

The tools will improve, and so will our ways of using them. For now, one of the most useful distinctions we make is between the response from an AI model and a conclusion that holds up under scrutiny.

AI helps us do both. The work of trust is deciding what to check, examining sources and assumptions, and judging whether a generated answer is sound enough to act upon. The final decision belongs to the people willing to stand behind the evidence.