Emulation

Neil D. Lawrence

GPSS, Sheffield

We may regard the present state of the universe as the effect of its past and the cause of its future. An intellect which at a certain moment would know all forces that set nature in motion, and all positions of all items of which nature is composed, …

… if this intellect were also vast enough to submit these data to analysis, it would embrace in a single formula the movements of the greatest bodies of the universe and those of the tiniest atom; for such an intellect nothing would be uncertain and the future just like the past would be present before its eyes.

— Pierre Simon Laplace (Laplace, 1814)

Game of Life

Loneliness

loneliness

Crowding

overcrowding

Birth

birth

Glider

Glider (1969)

Loafer

Loafer (2013)

\[ \text{data} + \text{model} \stackrel{\text{compute}}{\rightarrow} \text{prediction} \]

Laplace’s Gremlin

Philosophical Essay on Probabilities Laplace (1814) pg 5

Simulation System

Monolithic System

Service Oriented Architecture

Experiment, Analyze, Design

A Vision

We don’t know what science we’ll want to do in five years’ time, but we won’t want slower experiments, we won’t want more expensive experiments and we won’t want a narrower selection of experiments.

What do we want?

  • Faster, cheaper and more diverse experiments.
  • Better ecosystems for experimentation.
  • Data oriented architectures.
  • Data maturity assessments.
  • Data readiness levels.

Statistical Emulation

Emulation

Emulation

Emulation

Emulation

Emulation

Emukit

Javier Gonzalez Andrei Paleyes Mark Pullin Maren Mahsereci

Emukit

Emukit

Javier Gonzalez Andrei Paleyes Mark Pullin Maren Mahsereci

Modular Design

Introduce your own surrogate models.

from emukit.model_wrappers import GPyModelWrapper

To building your own model see this notebook.

from emukit.model_wrappers import YourModelWrapperHere

Structure

  • loop
  • model
  • candidate point calculator
  • acquisition
  • acquisition optimizer
  • user function
  • model updater
  • stopping condition

Emukit Vision

Emukit and Emulation

Methods

  • The different methods: Bayesian optimization, experimental design.

Models

  • The probabilistic model that will be used to emulate. Emukit doesn’t define these, the user brings their own.

Tasks

  • Still in development: High level goals that owners of the process/simulator might be actually interested in. Examples: measure quality of a simulator, explain complex system behavior.

Structure

while stopping condition is not met:
    optimize acquisition function
    evaluate user function
    update model with new observation

Loop

  • An abstract class where the different components come together.

Model

  • The surrogate model or emulator, often a Gaussian process.

Candidate Point Calculator

  • The routine that combines acquisition with optimizer to compute the next candidate point (or points).

Acquisition

  • Our acquisition function: in Bayesian Optimization, this might be Expected Improvement.

Acquisition Optimizer

  • The optimization routine we use to optimize the acquisition function. (often this is a non-linear optimizer like L-BFGS (Byrd et al., 1995))

User Function

  • The function we’re trying to reason about.

Model Updater

  • How to update our surrogate model when we have new training data.

Stopping Condition

  • How to decide when to stop our cycle of data acquisition from the target function.

Emukit Playground

Leah Hirst Cliff McCollum

Emukit Playground

Emukit Playground

Buying System

Monolithic System

Charlie Bell Peter Vosshall

Service Oriented Architecture

Charlie Bell Peter Vosshall

Question Mark Emails

Intellectual Debt

FIT Models to FIT Systems

  • Focus in machine learning has been on FAcT learning.
  • Fairness, accountability and Transparency in individual models.
  • But individual models aren’t the problem.
  • Fariness, interpetability and transparency required for whole system.

Thanks!

References

Byrd, R.H., Lu, P., Nocedal, J., 1995. A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific and Statistical Computing 16, 1190–1208.
Laplace, P.S., 1814. Essai philosophique sur les probabilités, 2nd ed. Courcier, Paris.