Take the Wheel: Managing AI Risk in Software Engineering Teams
By Ezra Buenk
Executive Summary
Unmonitored shadow AI is now pervasive in engineering teams, creating invisible risk as developers trade deep code understanding for short-term speed—resulting in a 77% maintenance failure rate when AI is removed and measurable technical debt in over 15% of AI commits. Despite near-universal adoption, real productivity gains remain flat at roughly 10%, with developers trusting only 30% of AI-generated output. The decisive factor is governance: structured AI use with guardrails preserves comprehension, catches vulnerabilities, and amplifies high-performing teams, while ungoverned organizations stall under accumulated debt. Closing the policy-to-practice loop with automated oversight turns AI risk into sustainable competitive advantage.
AI is transforming how technical teams build software, but adoption is outpacing oversight. In most organizations, no one knows which AI tools are in use or when, a phenomenon now common enough to earn its own label: shadow AI. Engineering teams are using generative tools without visibility from security, risk, or leadership, quietly degrading software quality and developer skills. The difference between organizations that thrive and those that stall comes down to governance: oversight, insight, and policy regarding AI is necessary to harness the productive potential of AI.
Cognitive Debt: Trading Speed for Understanding
AI is genuinely making some developers faster, but something is quietly going wrong. Research from Anthropic shows that developers are losing their capacity to understand the code they work with. Low-understanding AI users were slightly faster in the short term, confirming the trade-off between speed and depth.¹ Yet this is not a zero-sum game. Guided AI use proved to be the second-fastest approach while preserving deep understanding, suggesting that guardrails, not prohibition, are the answer.
A study on AI-scaffolded novice programming puts numbers to the risk. When AI was removed, unrestricted users suffered a 77% failure rate on subsequent maintenance tasks, compared with only 39% in the scaffolded group.² Developers owned the code legally, but not cognitively.² Introducing metacognitive guardrails dropped the failure rate to 44%.² The distinction is critical: cognitive offloading is not the same as cognitive outsourcing.
Technical Debt: The Last Few Percent
Unmonitored AI use also introduces material issues into the codebase. A large-scale empirical study found that more than 15% of AI commits show at least one code-quality issue, and 22.7% of those issues remained in repositories.³ Developers think they are shipping better code, but the data tells a different story.
Security research adds another layer of concern. While 87.9% of AI-generated code was safe, 12.1% contained vulnerabilities, with rates varying meaningfully across contexts.ā“ The danger lies in that last few percent slipping through undetected into production. Even a single security vulnerability getting through is enough to suffer meaningful damage: intellectual property theft, sensitive data leaks and regulatory fines. Think of it like a house: if you secure 4 out of 5 doors, any thief can still freely walk into the remaining door.
The Productivity Paradox
AI-authored code now accounts for more than one-quarter of all shipped production code,āµ yet average AI productivity growth remains flat at roughly 10%.āµ Gartner warns that tokenmaxxing (maximizing AI output volume) does not equal productivity.ā·
The disconnect is cultural as much as technical. According to the latest DORA report, 80% of developers believe they are more productive when using AI, yet only 30% trust the code.ā¶ In this context, DORA refers to the DevOps Research and Assessment institute: a Google-backed research program. The organizational impact is split: some companies saw customer-facing issues double, while others saw them halved.āµ AI alone does not determine the outcome.
Governance as the Multiplier
The difference between these outcomes is not the tool, but how AI is structured and governed.āµ A Google-backed study confirms that AI acts as an amplifier: it magnifies the strengths of high-performing organizations and the dysfunctions of struggling ones.ā¶ Organizations therefore need a clear AI policy and management tooling.āµ Gartner recommends concrete controls such as sandboxed environments, zero-trust architectures, and AI FinOps.ā·
Governance creates a competitive edge by ensuring that speed does not come at the cost of stability; while ungoverned teams eventually stall under the weight of AI-generated cognitive- and technical debt, governed organizations maintain a high release velocity with a clean, auditable and secure codebase that requires far less expensive remediation.
Speed is a liability if you are moving in the wrong direction.
Conclusion
Shadow AI, cognitive debt, and technical debt are not inevitable side effects of progress; they are warning signs of ungoverned acceleration. Closing the loop between board-level policy and code-level practice is what separates organizations that get held back by AI and the ones that sprint ahead. That loop requires tooling that is fast to deploy and easy to operate. Anove insAIght gets teams up and running in hours, not months, replacing manual compliance work with automated tracking for the EU AI Act, DORA, and more. Real-time visibility into shadow AI, vendor dependencies, and model risks are available from day one, alongside audit-ready reports that satisfy regulators without pulling engineers away from shipping. By embedding AI Management, Information Security, and Privacy Management into a single intuitive platform, you turn governance from a checkbox to a competitive advantage that sustains speed, without going in the wrong direction. Join us in our webinar to explore these issues in detail and to see how AI can be used responsibly.
¹ Anthropic (2026). How AI Impacts Skill Formation. 10.48550/arXiv.2601.20245
² Sankaranarayanan, S. (2026). Mitigating "Epistemic Debt" in Generative AI-Scaffolded Novice Programming using Metacognitive Scripts. 10.48550/arXiv.2602.20206.
³ Liu et al. (2026). Debt Behind the AI Boom. 10.48550/arXiv.2603.28592.
ā“ Schreiber, M. & Tippe, P. (2026). Security Vulnerabilities in AI-Generated Code: A Large-Scale Analysis of Public GitHub Repositories. 10.1007/978-981-95-3537-8_9.
āµ Tacho, L. (2026). Measuring Developer Productivity & AI Impact. DX Institute. As cited in Ivan Brezak Brkan (2026). This CTO Says 93% of Developers Use AI, but Productivity Is Still 10%. shiftmag.dev/this-cto-says-93-of-developers-use-ai-but-productivity-is-still-10-8013/
ā¶ DORA (2025). 2025 DORA Report: State of AI-Assisted Software Development. dora.dev/research/2025/dora-report/.
ā· Gartner (2026). Software Engineering AI Briefing: Tokenmaxxing and the AI Software Factory. gartner.com/document/G00854073.