
# The Three Ds of Machine Learning

Data Science Africa, Abuja

There are three types of lies: lies, damned lies and statistics

??

There are three types of lies: lies, damned lies and statistics

Benjamin Disraeli

There are three types of lies: lies, damned lies and statistics

Benjamin Disraeli 1804-1881

There are three types of lies: lies, damned lies and ‘big data’

Neil Lawrence 1972-?

### “Embodiment Factors”

See (“Living Together: Mind and Machine Intelligence” Lawrence, 2017a)(https://arxiv.org/abs/1705.07996)

### Effects

• This phenomenon has already revolutionised biology.
• Large scale data acquisition and distribution.
• Transcriptomics, genomics, epigenomics, ‘rich phenomics’.
• Great promise for personalized health.

### Societal Effects

• Automated decision making within the computer based only on the data.
• A requirement to better understand our own subjective biases to ensure that the human to computer interface formulates the correct conclusions from the data.
• Particularly important where treatments are being prescribed.
• But what is a treatment in the modern era: interventions could be far more subtle.

### Societal Effects

• Shift in dynamic from the direct pathway between human and data to indirect pathway between human and data via the computer
• This change of dynamics gives us the modern and emerging domain of data science

### What does Machine Learning do?

• Automation scales by codifying processes and automating them.
• Need:
• Interconnected components
• Compatible components
• Early examples:
• cf Colt 45, Ford Model T

### Codify Through Mathematical Functions

• How does machine learning work?
• Jumper (jersey/sweater) purchase with logistic regression

$\text{odds} = \frac{p(\text{bought})}{p(\text{not bought})}$ $\log \text{odds} = \beta_0 + \beta_1 \text{age} + \beta_2 \text{latitude}.$

### Codify Through Mathematical Functions

• How does machine learning work?
• Jumper (jersey/sweater) purchase with logistic regression

$p(\text{bought}) = \sigmoid{\beta_0 + \beta_1 \text{age} + \beta_2 \text{latitude}}.$

### Codify Through Mathematical Functions

• How does machine learning work?
• Jumper (jersey/sweater) purchase with logistic regression

$p(\text{bought}) = \sigmoid{\boldsymbol{\beta}^\top \inputVector}.$

### Codify Through Mathematical Functions

• How does machine learning work?
• Jumper (jersey/sweater) purchase with logistic regression

$\dataScalar = \mappingFunction\left(\inputVector, \boldsymbol{\beta}\right).$

We call $$\mappingFunction(\cdot)$$ the prediction function.

### Fit to Data

• Use an objective function

$\errorFunction(\boldsymbol{\beta}, \dataMatrix, \inputMatrix)$

• E.g. least squares $\errorFunction(\boldsymbol{\beta}, \dataMatrix, \inputMatrix) = \sum_{i=1}^\numData \left(\dataScalar_i - \mappingFunction(\inputVector_i, \boldsymbol{\beta})\right)^2.$

### Two Components

• Prediction function, $$\mappingFunction(\cdot)$$
• Objective function, $$\errorFunction(\cdot)$$

$\text{data} + \text{model} \xrightarrow{\text{compute}} \text{prediction}$

 $\text{data} + \text{model} \xrightarrow{\text{compute}} \text{prediction}$

### Machine Learning in Supply Chain

Containerization has had a dramatic effect on global economics, placing many people in the developing world at the end of the supply chain.

### Machine Learning in Supply Chain

• Supply chain: Large Automated Decision Making Network
• Amazon’s supply chain: Possibly the world’s largest ‘AI’
• Major Challenge:
• We have a mechanistic understanding of supply chain.
• Machine learning is a data driven technology.

### The Tribal Mentality

• $$\text{data} + \text{model}$$ is not new.
• Dates back to Newton, Laplace, Gauss
• Plethora of fields: E.g.
• Operations Research
• Control
• Econometrics
• Statistics
• Machine learning
• Data science

### The Tribal Mentality

• This can lead to confusion:
• Born in different eras
• Driven by different motivations
• Arrive at different solutions

### Tribalism Can be Good

• Allows for consensus on best practice.
• Shared set of goals
• Ease of commiunication
• Rapid deployment of robust solutions

### Professional Tribes

• This is the nature of professions
• lawyers
• medics
• doctors
• engineers
• accountants

### Different Views

$\text{data} + \text{model}$

• For OR, control, stats etc.
• More things unite us rather than divide us.

### We’re no longer hunter gatherers …

• The automation challenges we face require
• all of our best ideas.
• rethinking what $$\text{data}+\text{model}$$ means
• rapid deployment and continuous monitoring
• This is the era of data science

### Discomfort and Disconformation

• Talking across field boundaries is critical.
• It helps us disconfirm our beliefs.
• It’s not comfortable, but it’s vital.

### Challenges

1. Paradoxes of the Data Society

2. Quantifying the Value of Data

3. Privacy, loss of control, marginalization

• Able to quantify to a greater and greater degree the actions of individuals

• But less able to characterize society

• As we measure more, we understand less

### What?

• Perhaps greater preponderance of data is making society itself more complex
• Therefore traditional approaches to measurement are failing
• Curate’s egg of a society: it is only ‘measured in parts’

• Modern Measurement deals with depth (many subjects) … or breadth lots of detail about subject.
• Can deal with large or large
• roughly equal to ?
• Stratification of populations: batch effects etc.

### Wood or Tree

• Can either see a wood or a tree.

### Examples

• Election polls (UK 2015 elections, EU referendum, US 2016 elections)
• Clinical trials vs personalized medicine: Obtaining statistical power where interventions are subtle. e.g. social media

### Challenges

• Social media memes
• Filter bubbles and echo chambers

### Solutions

• More classical statistics!
• Like the ‘paperless office’
• A better characterization of human needs and flaws
• Larger studies (100,000 genome)

### Quantifying the Value of Data

There’s a sea of data, but most of it is undrinkable.

We require data-desalination before it can be consumed!

### Data

• 90% of our time is spent on validation and integration (Leo Anthony Celi)
• “The Dirty Work We Don’t Want to Think About” (Eric Xing)
• “Voodoo to get it decompressed” (Francisco Giminez)
• In health care clinicians collect the data and often control the direction of research through guardianship of data.

### Value

• How do we measure value in the data economy?
• How do we encourage data workers: curation and management
• Incentivization for sharing and production.
• Quantifying the value in the contribution of each actor.

{

### Credit Allocation

• Direct work on data generates an enormous amount of ‘value’ in the data economy but this is unaccounted in the economy

• Hard because data is difficult to ‘embody’

• Value of shared data: Wellcome Trust 2010 Joint Statement (from the “Foggy Bottom” meeting)

https://arxiv.org/pdf/1705.02245.pdf Data Readiness Levels (Lawrence, 2017b)

• Hearsay data.
• Availability, is it actually being recorded?
• privacy or legal constraints on the accessibility of the recorded data, have ethical constraints been alleviated?
• Format: log books, PDF …
• limitations on access due to topology (e.g. it’s distributed across a number of devices)
• At the end of Grade C data is ready to be loaded into analysis software (R, SPSS, Matlab, Python, Mathematica)

• faithfulness and representation
• visualisations.
• exploratory data analysis
• noise characterisation.

• Missing values.
• Schema alignment, record linkage, data fusion
• Example:

• Compare with classical statistics:
• Classically: question is first data comes later.
• Today: data is first question comes later.

### Data First

In a data first company teams own their data quality issues at least as far as grade B1.

• The usability of data
• Consider appropriateness of a given data set to answer a particular question or to be subject to a particular analysis.

### Recursive Effects

• Grade A may also require:
• data integration
• active collection of new data.
• rebalancing of data to ensure fairness
• annotation of data by human experts
• revisiting the collection (and running through the appropriate stages again)

### A1 Data

• A1 data is ready to make available for challenges or AutoML platforms.

### Also …

• Encourage greater interaction between application domains and data scientists
• Encourage visualization of data

• Data Joel Tests

### Solutions

• Encourage greater interaction between application domains and data scientists
• Encourage visualization of data
• Implications for incentivization schemes

### Privacy, Loss of Control and Marginalization

• Society is becoming harder to monitor
• Individual is becoming easier to monitor

### Hate Speech or Political Dissent?

• social media monitoring for ‘hate speech’ can be easily turned to political dissent monitoring

### Marketing

• can become more sinister when the target of the marketing is well understood and the (digital) environment of the target is also so well controlled

### Free Will

• What does it mean if a computer can predict our individual behavior better than we ourselves can?

### Discrimination

• Potential for explicit and implicit discrimination on the basis of race, religion, sexuality, health status
• All prohibited under European law, but can pass unawares, or be implicit
• GDPR: General Data Protection Regulation

{Discrimination {.slide: data-transition=“none”}

• Potential for explicit and implicit discrimination on the basis of race, religion, sexuality, health status

• All prohibited under European law, but can pass unawares, or be implicit

• GDPR: Good Data Practice Rules

### Marginalization

• Credit scoring, insurance, medical treatment
• What if certain sectors of society are under-represented in our analysis?
• What if Silicon Valley develops everything for us?

### Amelioration

• Work to ensure individual retains control of their own data
• We accept privacy in our real lives, need to accept it in our digital
• Control of persona and ability to project
• Need better technological solutions: trust and algorithms.

### The Three Ds of Machine Learning Systems Design

• Three primary challenges of Machine Learning Systems Design.
1. Decomposition
2. Data
3. Deployment

### Decomposition

• ML is not Magical Pixie Dust.
• It cannot be sprinkled thoughtlessly.
• We cannot simply automate all decisions through data

### Decomposition

We are constrained by:

1. Our data.
2. The models.

• Careful thought needs to be put into sub-processes of task.
• Any repetitive task is a candidate for automation.

### Pigeonholing

1. Can we decompose decision we need to repetitive sub-tasks where inputs and outputs are well defined?
2. Are those repetitive sub-tasks well represent by a mathematical mapping?

### A Trap

• Over emphasis on the type of model we’re deploying.
• Under emphasis on the appropriateness of the task decomposition.

### Co-evolution

• Absolute decomposition is impossible.
• If we deploy a weak component in one place, downstream system will compensate.
• Systems co-evolve … there is no simple solution
• Trade off between performance and decomposability.
• Need to monitor deployment

### Data

• Hard to overstate its importance.
• Half the equation of $$\text{data} + \text{model}$$.
• Often utterly neglected.

### Data Neglect

• Arises for two reasons.
1. Data cleaning is perceived as tedious.
2. Data cleaning is complex.

### Data Cleaning

• Seems difficult to formulate into readily teachable princples.
• Heavily neglected in data science, statistics and ML courses.
• In practice most scientists spend around 80% of time data cleaning.

### The Data Crisis

The major cause of the software crisis is that the machines have become several orders of magnitude more powerful! To put it quite bluntly: as long as there were no machines, programming was no problem at all; when we had a few weak computers, programming became a mild problem, and now we have gigantic computers, programming has become an equally gigantic problem.

Edsger Dijkstra (1930-2002), The Humble Programmer

### The Data Crisis

The major cause of the data crisis is that machines have become more interconnected than ever before. Data access is therefore cheap, but data quality is often poor. What we need is cheap high quality data. That implies that we develop processes for improving and verifying data quality that are efficient.

There would seem to be two ways for improving efficiency. Firstly, we should not duplicate work. Secondly, where possible we should automate work.

Me

### Data Science as Debugging

• Analogies: For Software Engineers describe data science as debugging.

### Premise

Our machine learning is based on a software systems view that is 20 years out of date.

• Deployment of modeling code.
• Data dependent models in production need continuous monitoring.

### Continuous Monitoring

• Continuous deployment:
• We’ve changed the code, we should test the effect.
• Continuous Monitoring:
• The world around us is changing, we should monitor the effect.
• Update our notions of testing: progression testing

### Data Orientated Architectures

• Historically we’ve been software first
• A necessary but not sufficient condition for data first
• Move from
1. software orientated architectures
2. data orientated architectures

### Streaming Architectures

• AWS Kinesis, Apache Kafka
• Nodes in the architecture are stateless
• They persist through storing state on streams
• This brings the data inside out

### Outlook for Machine Learning

• Risen to prominence to scale our activities.
• To scale activities need more computer based automation.
• Machine learning allows us to automate processes previously out of reach.

### Conclusion

• Technologically evolving environment.
• ML is a key component of decision making.
• Data is the key component of ML.
• ML is critically dependent on data.
• Challenges in system decomposition, data curation and model deployment.

### References

Lawrence, N.D., 2017a. Living together: Mind and machine intelligence. arXiv.

Lawrence, N.D., 2017b. Data readiness levels. arXiv.