Technical debt is incurred when complex systems are rapidly deployed without due thought as to how they will be maintained. Intellectual debt is incurred when complex systems are rapidly deployed without due thought to how they’ll be explained. Both problems are pervasive in the design and deployment of large scale algorithmic decision making engines. In this talk we’ll review the origin of the problem, and propose a roadmap for obtaining solutions. It’s a journey that will require collaboration between industry, academia, third sector, and government.
The Great AI Fallacy
I don’t know what Artificial Intelligence means, but one commonality I see in the use of the term is an idea which I’m coming to think of as “The Great AI Fallacy”.
The fallacy is associated with an implicit promise that is embedded in many statements about Artificial Intelligence. Artificial Intelligence, as it currently exists, is merely a form of automated decision making. The implicit promise of Artificial Intelligence is that it will be the first wave of automation where the machine adapts to the human, rather than the human adapting to the machine.
How else can we explain the suspension of sensible business judgment that is accompanying the hype surrounding AI?
This fallacy is particularly pernicious because there are serious benefits to society in deploying this new wave of data-driven automated decision making. But the AI Fallacy is causing us to suspend our calibrated skepticism that is needed to deploy these systems safely and efficiently.
The problem is compounded because many of the techniques that we’re speaking of were originally developed in academic laboratories in isolation from real-world deployment.
Lean Startup Methodology
The Mythical Man-month
Separation of Concerns
Technical debt is the inability to maintain your complex software system.
Intellectual debt is the inability to explain your software system.
Virtual Gemba Walks
- ML Models deployed as ‘regular software’
- Sanity checks are being suspended
- Models are driven by “average case”
The Death of the Programmer
FIT Models to FIT Systems
The Census and the Big Data Paradox
Brooks, Frederick. n.d. The Mythical Man-Month. Addison-Wesley.