In this talk we introduce a five year project funded by the UK’s Turing Institute to shift the focus from developing AI systems to deploying AI systems that are safe and reliable. The AI systems we are developing and deploying are based on interconnected machine learning components. There is a need for AI-assisted design and monitoring of these systems to ensure they perform robustly, safely and accurately in their deployed environment. We address the entire pipeline of AI system development, from data acquisition to decision making. Data Oriented Architectures are an ecosystem that includes system monitoring for performance, interpretability and fairness. The will enable us to move from individual component optimisation to full system monitoring and optimisation.
The promise of the fourth industrial revolution is that this wave of automation is the first wave of automation where the machines adapt to serve us rather than us adapting to serve the machine.
That promise will remain unfulfilled with our current approach to systems design.
This proposal is about addressing that gap, but to first understand the gap, let’s look at comparisons between the approach we take to systems design, and the way that natural systems evolve.
Artificial Intelligence (AI) solutions are based on machine learning algorithms (ML), but each ML solution is only capable of solving a restricted task, e.g. a supervised learning problem. Consequently, any AI that we deploy today takes the form of an ML System with interacting components. As these ML systems become larger and more complex, challenges in interpretation, explanation, accuracy and fairness arise. This project addresses these issues. The challenges include (Lawrence 2019): the decomposition of the system, the data availability, and the performance of the system in deployment. Collectively we refer to these challenges as the “Three Ds of ML Systems Design”.
It used to be true that computers only did what we programmed them to do, but today AI systems are learning from our data. This introduces new problems in how these systems respond to their environment.
We need to better monitor how data is influencing decision making and take corrective action as required.
Aim
Our aim is to scale our ability to deploy safe and reliable AI solutions. Our technical approach is to do this through data-oriented software engineering practices and deep system emulation. We will do this through a significant extension of the notion of Automated ML (AutoML) to Automated AI (AutoAI), this relies on a shift from Bayesian Optimisation to Bayesian System Optimisation. The project will develop a toolkit for automating the deployment, maintenance and monitoring of artificial intelligence systems.
The notion relates from the “Promise of AI”: it promises to be the first generation of automation technology that will adapt to us, rather than us adapting to it. The premise of the project is that this promise will remain unfulfilled with current approaches to systems design and deployment.
Figure: SafeBoda is a ride allocation system for Boda Boda drivers. Let’s imagine the capabilities we need for such an AI system.
SafeBoda is a Kampala based rider allocation system for Boda Boda drivers. Boda boda are motorcycle taxis which give employment to, often young men, across Kampala. Safe Boda is driven by the knowledge that road accidents are set to match HIV/AIDS as the highest cause of death in low/middle income families by 2030.
With road accidents set to match HIV/AIDS as the highest cause of death in low/middle income countries by 2030, SafeBoda’s aim is to modernise informal transportation and ensure safe access to mobility.
SafeBoda and other projects like Kudu provide us with our motivating examples. Our aim is to create an ecosystem for machine learing system deployment that minimises the operational load. Ideally, we would like complex AI systems to be maintainable by a small team, e.g. two people, with Masters-level education from the institutions that host Data Science Africa (e.g. Ashesi University, Makerere University, Dedan Kimathi University of Technology, AUST, AIST, Addis Ababa).
As of 24th October 2019, the Turing Institute announced that this work has been funded through an Alan Turing Institute Senior AI Fellowship. This is the first Senior AI fellowship and it provides funding for five years.
There is no way that the team we’re building will be able to deliver on this agenda alone, so please join us in addressing these challenges!
Figure: Some software components in a ride allocation system. Circled components are hypothetical, rectangles represent actual data.
Currently, our main approach to systems design involves designing a system in a component-wise manner. Attempts to replicate the capabilities of evolved systems through specifying the objective, rather than evolving behaviour.
This gives vulnerabilities that we are exposing to the natural environment. Many security problems that we face today are the result of bugs that mean that code and data are not separate in thee systems we deploy, imagine what will happen when we deploy systems that purposefully short-circuit this protection into uncontrolled environments.
The Three Ds of Machine Learning Systems Design [edit]
We can characterize the challenges for integrating machine learning within our systems as the three Ds. Decomposition, Data and Deployment.
The next wave of machine learning is a paradigm shift in the way we think about computer science.
Classical computer science assumes that ‘data’ and ‘code’ are separate, and this is the foundation of secure computer systems. In machine learning, ‘data’ is ‘software’, so the decision making is directly influenced by the data. We are short-circuiting a fundamental assumption of computer science, we are breeching the code/data separation.
This means we need to revisit many of our assumptions and tooling around the machine learning process. In particular, we need new approaches to systems design, new approaches to programming languages that highlight the importance of data, and new approaches to systems security.
We introduce the notion of Bayesian system optimisation. Standard Bayesian optimisation is about optimising individual components under a given (localised) optimisation criterion. Bayesian system optimisation is about realising that there are upstream and downstream effects, ‘no model is an island’. If we can use emulation to estimate those effects, then we can optimise individual components not just according to their own objective functions, but according to their situation in the wider system and their downstream effects.
Supervised machine learning models are data-driven statistical functional estimators. Each ML model is trained to perform a task. Machine learning systems are created when these models are integrated as interacting components in a more complex system that carries out a larger scale task, e.g. an autonomous drone delivery system.
Artificial Intelligence can also be seen as algorithmic decision-making. ML systems are data driven algorithmic decision-makers. Designing decision-making engines requires us to firstly decompose the system into its component parts. The decompositions are driven by (1) system performance requirements (2) the suite of ML algorithms at our disposal (3) the data availability. Performance requirements could be computational speed, accuracy, interpretability, and ‘fairness’. The current generation of ML Systems is often based around supervised learning and human annotated data. But in the future, we may expect more use of reinforcement learning and automated knowledge discovery using unsupervised learning.
The classical systems approach assumes decomposability of components. In ML, upstream components (e.g. a pedestrian detector in an autonomous vehicle) make decisions that require revisiting once a fuller picture is realized at a downstream stage (e.g. vehicle path planning). The relative weaknesses and strengths of the different component parts need to be assessed when resolving conflicts.
In long-term planning, e.g. logistics and supply chain, a plan may be computed multiple times under different constraints as data evolves. In logistics, an initial plan for delivery may be computed when an item is viewed on a webpage. Webpage waiting-time constraints dominate the solution we choose. However, when an order is placed the time constraint may be relaxed and an accuracy constraint or a cost constraint may now dominate.
Such sub-systems will make inconsistent decisions, but we should monitor and control the extent of the inconsistency.
One solution to aid with both the lack of decomposability of the components and the inconsistency between components is end-to-end learning of the system. End-to-end learning is when we use ML techniques to fit parameters across the entire decision pipeline. We exploit gradient descent and automated differentiation software to achieve this. However, components in the system may themselves be running a simulation (e.g. a transport delivery-time simulation) or optimization (e.g. a linear program) as a subroutine. This limits the universality of automatic differentiation. Another alternative is to replace the entire system with a single ML model, such as in Deep Reinforcement Learning. However, this can severely limit the interpretability of the resulting system.
We envisage AutoAI as allowing us to take advantage of end-to-end learning without sacrificing the interpretability of the underlying system. Instead of optimizing each component individually, we introduce Bayesian system optimization (BSO). We will make use of the end-to-end learning signals and attribute them to the system sub-components through the construction of an interconnected network of surrogate models, known as emulators, each of which is associated with an individual component from the underlying ML-system. Instead of optimizing each component individually (e.g. by classical Bayesian optimization) in BSO we account for upstream and downstream interactions in the optimization, leveraging our end-to-end knowledge without damaging the interpretability of the underlying system.
We operate in a technologically evolving environment. Machine learning is becoming a key coponent in our decision-making capabilities, our intelligence and strategic command. However, technology drove changes in battlefield strategy. From the stalemate of the first world war to the tank-dominated Blitzkrieg of the second, to the asymmetric warfare of the present. Our technology, tactics and strategies are also constantly evolving. Machine learning is part of that evolution solution, but the main challenge is not to become so fixated on the tactics of today that we miss the evolution of strategy that the technology is suggesting.
Data oriented programming offers a set of development methodologies which ensure that the system designer considers what decisions are required, how they will be made, and critically, declares this within the system architecture.
This allows for monitoring of data quality, fairness, model accuracy and opens the door to Auto AI: a more sophisticated form of auto ML where full redployments of models are considered while analyzing the information dynamics of a complex automated decision-making system.
References
Lawrence, Neil D. 2019. “Data Science and Digital Systems: The 3Ds of Machine Learning Systems Design.” arXiv. https://arxiv.org/abs/1903.11241.
