To stay competitive, the U.S. should evaluate AI tools like large language models based on performance, not explainability. Trust should be grounded in results, not unrealistic expectations of human-like reasoning.
The United States Must Treat AI as a Strategic Asset in Great Power Competition
As the United States enters a new era of great power competition—particularly with a technologically ambitious China—questions about how and when to trust artificial intelligence systems like large language models (LLMs) are not merely technical. They are strategic. These tools will increasingly shape how the United States allocates resources, prioritizes defense investments, and maintains a credible military posture in the Indo-Pacific and beyond.
Focusing on Explainability Could Undermine Strategic Adoption of AI
LLMs are not reasoning agents. They are pattern recognizers trained on vast datasets, designed to predict the next word in a sequence. Like a chess grandmaster making a brilliant but intuitive move, LLMs often cannot explain why they generate a specific output. Yet the Department of Defense, through organizations like the Chief Digital and AI Office, has prioritized explainable AI as a requirement for operational use. This well-meaning mandate risks missing the point.
Explainability in LLMs may not be technically achievable—and chasing it could be a strategic distraction. These models don’t “understand” in the human sense. Their outputs are statistical associations, not causal conclusions. Post-hoc explanations, while satisfying, can be misleading and ultimately hinder adoption of tools that could enhance strategic foresight, intelligence analysis, and operational planning.
The real danger lies in overemphasizing explainability at the expense of performance. Many decisions in national security—from target selection to long-range procurement—already involve opaque but proven processes, like wargaming or expert judgment. LLMs, if properly tested, can complement these approaches by processing volumes of information at speeds that human analysts cannot match.
Rather than trying to make LLMs more “human,” we should evaluate them using criteria aligned with how they actually function: consistency, accuracy, and clarity about limitations. We should ask:
- Was the model trained on credible, relevant data?
- How often does it produce accurate outputs under stress-tested conditions?
- What risks—bias, hallucination, data contamination—are inherent in its use?
Emerging methods like automated fact-checking have reduced hallucination rates dramatically—from around 9% to 0.3% in some models. Performance-based frameworks, like TrustLLM, show promise for assessing model reliability more wholistically than explanations ever could.
Trust in AI Should Be Earned Through Consistent Results, Not Human-Like Explanations
To ensure effective and safe integration of large language models (LLMs) in military and defense contexts, policymakers should prioritize operational testing over explainability mandates. Rather than focusing on artificial interpretability, systems should be evaluated against performance thresholds before deployment. This approach emphasizes empirical reliability and ensures that AI tools deliver consistent, verifiable results under real-world conditions.
Policymakers must also educate military leaders on the nature and limitations of LLMs. Trust in these models should stem from measurable outcomes, not from an illusion of understanding or human-like qualities. As non-sentient tools, LLMs operate through pattern recognition, not cognition, and should not be expected to mimic human reasoning or self-awareness.
Finally, it is essential to develop AI adoption guidelines tailored to specific use cases. Different operational scenarios require different levels of oversight and reliability. For instance, intelligence summarization may prioritize high consistency, whereas warfighting applications could necessitate enhanced safeguards and persistent human-in-the-loop controls to mitigate risks and ensure accountability.
Overall, trust in LLMs should be rooted not in their ability to sound human, but in their consistent capacity to produce accurate, repeatable, and auditable outputs. Treating them as digital oracles is both unrealistic and counterproductive. Evaluating AI systems based on performance, rather than interpretability or anthropomorphic appeal, is a far more pragmatic and effective approach.
About the Author: Michael Perry
Lt Col Michael “Sparky” Perry is an Air Force officer and MC-130 Command Pilot with Master’s degrees in Business Administration and Military Art & Science. He is currently a National Defense Fellow at the Sam Nunn School of International Affairs at the Georgia Institute of Technology. In coordination with several Department of Defense and academic institutions, his 2024/5 Defense Fellow research focuses on artificial intelligence for DoD decision-support and the trustworthiness of large language models.
Image: Shutterstock
