Emulation

We may regard the present state of the universe as the effect of its past and the cause of its future. An intellect which at a certain moment would know all forces that set nature in motion, and all positions of all items of which nature is composed, …

… if this intellect were also vast enough to submit these data to analysis, it would embrace in a single formula the movements of the greatest bodies of the universe and those of the tiniest atom; for such an intellect nothing would be uncertain and the future just like the past would be present before its eyes.

— Pierre Simon Laplace (Laplace, 1814)

Game of Life

Loneliness

loneliness

Crowding

overcrowding

Birth

birth

Glider

Glider (1969)

Loafer

\[ \text{data} + \text{model} \stackrel{\text{compute}}{\rightarrow} \text{prediction} \]

Laplace’s Gremlin

Philosophical Essay on Probabilities Laplace (1814) pg 5

Simulation System

Monolithic System

Service Oriented Architecture

Experiment, Analyze, Design

A Vision

We don’t know what science we’ll want to do in five years’ time, but we won’t want slower experiments, we won’t want more expensive experiments and we won’t want a narrower selection of experiments.

What do we want?

• Faster, cheaper and more diverse experiments.
• Better ecosystems for experimentation.
• Data oriented architectures.
• Data maturity assessments.
• Data readiness levels.

Statistical Emulation

Emulation

Emulation

Emulation

Emulation

Emulation

Emukit

Emukit

Emukit

• Led by: Javier Gonzalez and Andrei Paleyes
  • Team: Mark Pullin, Maren Mahsereci, Alex Gessner, Aaron Klein, Henry Moss and David-Elias Künstle.
  • Management: Cliff McCollum & Neil
• Available on Github
  • Example sensitivity notebook, documentation https://emukit.readthedocs.io/en/latest/

Modular Design

Introduce your own surrogate models.

from emukit.model_wrappers import GPyModelWrapper

To building your own model see this notebook.

from emukit.model_wrappers import YourModelWrapperHere

Structure

• loop
• model
• candidate point calculator
• acquisition
• acquisition optimizer
• user function
• model updater
• stopping condition

Emukit Vision

Emukit and Emulation

Methods

• The different methods: Bayesian optimization, experimental design.

Models

• The probabilistic model that will be used to emulate. Emukit doesn’t define these, the user brings their own.

Tasks

• Still in development: High level goals that owners of the process/simulator might be actually interested in. Examples: measure quality of a simulator, explain complex system behavior.

Structure

while stopping condition is not met:
    optimize acquisition function
    evaluate user function
    update model with new observation

Loop

• An abstract class where the different components come together.

Model

• The surrogate model or emulator, often a Gaussian process.

Candidate Point Calculator

• The routine that combines acquisition with optimizer to compute the next candidate point (or points).

Acquisition

• Our acquisition function: in Bayesian Optimization, this might be Expected Improvement.

Acquisition Optimizer

• The optimization routine we use to optimize the acquisition function. (often this is a non-linear optimizer like L-BFGS (Byrd et al., 1995))

User Function

• The function we’re trying to reason about.

Model Updater

• How to update our surrogate model when we have new training data.

Stopping Condition

• How to decide when to stop our cycle of data acquisition from the target function.

Emukit Playground

• Work Leah Hirst, Software Engineering Intern and Cliff McCollum.

• Tutorial on emulation.

Emukit Playground

Emukit Playground

Buying System

Monolithic System

Service Oriented Architecture

Question Mark Emails

Intellectual Debt

FIT Models to FIT Systems

• Focus in machine learning has been on FAcT learning.
• Fairness, accountability and Transparency in individual models.
• But individual models aren’t the problem.
• Fariness, interpetability and transparency required for whole system.