Eye Fitting Straight Lines in the Modern Era

class:title-slide-custom

<style>

/* colors: #EEB422, #8B0000, #191970, #00a8cc */
/* define the new color palette here! */
a, a > code {
  color: #8B0000;
  text-decoration: none;
}

.title-slide h2::after, .mline h1::after {
  content: '';
  display: block;
  border: none;
  background-color: #8B0000;
  color: #8B0000;
  height: 2px;
}

.remark-slide-content {
  background-color: #FFFFFF;
  border-top: 80px solid #8B0000;
  font-size: 20px;
  font-weight: 300;
  line-height: 1.5;
  
  background-image: url(css/UNL.svg);
	background-position: 2% 98%;
	background-size: 10%;
	border-bottom: 0;
}

.inverse {
  background-color: #8B0000;
  
  
	
  
}

.remark-slide-content > h1 {
  font-family: 'Roboto';
  font-weight: 300;
  font-size: 45px;
  margin-top: -95px;
  margin-left: -00px;
  color: #FFFFFF;
}

.title-slide {
  background-color: #FFFFFF;
  
  background-image:  url(css/UNL.svg);
	background-position: 98% 98%;
	
	background-size: 20%;
	border-bottom: 0;
	border: 10px solid 	#8B0000;
	
}

.title-slide > h1  {
  color: #111111;
  font-size: 32px;
  text-shadow: none;
  font-weight: 500;
  text-align: left;
  margin-left: 15px;
  padding-top: 80px;
}
.title-slide > h2  {
  margin-top: -25px;
  padding-bottom: -20px;
  color: #111111;
  text-shadow: none;
  font-weight: 100;
  font-size: 28px;
  text-align: left;
  margin-left: 15px;
}
.title-slide > h3  {
  color: #111111;
  text-shadow: none;
  font-weight: 100;
  font-size: 28px;
  text-align: left;
  margin-left: 15px;
  margin-bottom: -20px;
}

body {
  font-family: 'Roboto';
  font-weight: 300;
}

.remark-slide-number {
  font-size: 13pt;
  font-family: 'Roboto';
  color: #272822;
  opacity: 1;
}

.inverse .remark-slide-number {
  font-size: 13pt;
  font-family: 'Roboto';
  color: #FAFAFA;
  opacity: 1;
}

.title-slide-custom .remark-slide-number {
  display: none;
}

.title-slide-custom h3::after, .mline h1::after {
  content: '';
  display: block;
  border: none;
  background-color: #8B0000;
  color: #8B0000;
  height: 2px;
}

.title-slide-custom {
  background-color: #FFFFFF;
  
  background-image:  url(css/UNL.svg);
	background-position: 98% 98%;
	
	background-size: 20%;
	border-bottom: 0;
	border: 10px solid 	#8B0000;
	
}

.title-slide-custom > h1  {
  color: #111111;
  font-size: 40px;
  text-shadow: none;
  font-weight: 500;
  text-align: left;
  margin-left: 15px;
  padding-top: 80px;
  padding-bottom: 10px;
}
.title-slide-custom > h2  {
  margin-top: -25px;
  padding-bottom: 30px;
  color: #111111;
  text-shadow: none;
  font-weight: 100;
  font-size: 32px;
  text-align: left;
  margin-left: 15px;
}
.title-slide-custom > h3  {
  margin-top: -25px;
  padding-bottom: -25px;
  color: #111111;
  text-shadow: none;
  font-weight: 100;
  font-size: 32px;
  text-align: left;
  margin-left: 15px;
}
.title-slide-custom > h4  {
  color: #111111;
  text-shadow: none;
  font-weight: 100;
  font-size: 28px;
  text-align: left;
  margin-left: 15px;
  margin-bottom: -30px;
  padding-bottom: -25px;
}
.title-slide-custom > h5  {
  color: #111111;
  text-shadow: none;
  font-weight: 100;
  font-size: 24px;
  text-align: left;
  margin-left: 15px;
  margin-bottom: -40px;
}

</style>

<br><br><br>
# Eye Fitting Straight Lines in the Modern Era
## Midwest Women in Science Conference
### September 18, 2021
#### Emily A. Robinson
#### Department of Statistics, University of Nebraska - Lincoln
##### <svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:black;overflow:visible;position:relative;"><path d="M502.3 190.8c3.9-3.1 9.7-.2 9.7 4.7V400c0 26.5-21.5 48-48 48H48c-26.5 0-48-21.5-48-48V195.6c0-5 5.7-7.8 9.7-4.7 22.4 17.4 52.1 39.5 154.1 113.6 21.1 15.4 56.7 47.8 92.2 47.6 35.7.3 72-32.8 92.3-47.6 102-74.1 131.6-96.3 154-113.7zM256 320c23.2.4 56.6-29.2 73.4-41.4 132.7-96.3 142.8-104.7 173.4-128.7 5.8-4.5 9.2-11.5 9.2-18.9v-19c0-26.5-21.5-48-48-48H48C21.5 64 0 85.5 0 112v19c0 7.4 3.4 14.3 9.2 18.9 30.6 23.9 40.7 32.4 173.4 128.7 16.8 12.2 50.2 41.8 73.4 41.4z"/></svg> [emily.robinson@huskers.unl.edu](emily.robinson@huskers.unl.edu)
##### <svg aria-hidden="true" role="img" viewBox="0 0 576 512" style="height:1em;width:1.12em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:black;overflow:visible;position:relative;"><path d="M280.37 148.26L96 300.11V464a16 16 0 0 0 16 16l112.06-.29a16 16 0 0 0 15.92-16V368a16 16 0 0 1 16-16h64a16 16 0 0 1 16 16v95.64a16 16 0 0 0 16 16.05L464 480a16 16 0 0 0 16-16V300L295.67 148.26a12.19 12.19 0 0 0-15.3 0zM571.6 251.47L488 182.56V44.05a12 12 0 0 0-12-12h-56a12 12 0 0 0-12 12v72.61L318.47 43a48 48 0 0 0-61 0L4.34 251.47a12 12 0 0 0-1.6 16.9l25.5 31A12 12 0 0 0 45.15 301l235.22-193.74a12.19 12.19 0 0 1 15.3 0L530.9 301a12 12 0 0 0 16.9-1.6l25.5-31a12 12 0 0 0-1.7-16.93z"/></svg> [www.emilyarobinson.com](https://www.emilyarobinson.com/)
##### <svg aria-hidden="true" role="img" viewBox="0 0 496 512" style="height:1em;width:0.97em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:black;overflow:visible;position:relative;"><path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"/></svg> [earobinson95](https://github.com/earobinson95)
<br><br>
.medium[*Slides: https://bit.ly/3EgSVU4*]

???

Thank you, everyone for coming! I am a PhD candidate in the Department of Statistics at the University of Nebraska - Lincoln. I will be presenting my research titled, Eye Fitting Straight Lines in the Modern Era.

---
class:primary
# Introduction to Graphics

Data visualization is defined as the art of drawing **graphical charts** in order to display data .

**What are graphics useful for?** (Lewandowsky and Spence, 1989)
+ Data cleaning.
+ Exploring data structure.
+ Communicating information.

**Who uses graphics?**
+ Governments (Harms, 1991; Playfair, 1801; Walker, 2013).
+ Companies (Chandar, Collier, and Miranti, 2012; Yates, 1985).
+ News sources and mass media (Aisch, Cohn, Cox, et al., 2016).
+ Scientific publications (Gouretski and Koltermann, 2007).

???
To get started, we are first going to lay the foundation of graphics. Data visualization has become central tool in modern data science and statistics. Unwin 2020 defines data visualization as the art of drawing graphical charts in order to display data.

Graphics are useful for data cleaning, exploring data structure, and communicating information.

Graphics are used by governments, companies, news sources, mass media, and in scientific publications.

Despite the popularity of graphics, we are too accepting of them as default without asking critical questions about the graphics we create or view (Unwin, 2020). We must begin asking ourselves **How effective is this graph at communicating useful information?**

---
class:primary
# Testing Statistical Graphics

Evaluate design choices and understand cognitive biases through the use of **visual tests**.

Could ask participants to:

- identify differences in graphs.
- read information off of a chart accurately. 
- use data to make correct real-world decisions.
- predict the next few observations.

???
One way we can evaluate these design choices through the use of graphical tests.

Could ask participants to:

- identify differences in graphs.
- read information off of a chart accurately.
- use data to make correct real-world decisions.
- predict the next few observations.

All of these types of tests require different levels of use and manipulation of the information presented in the chart.

.pull-left[
**Lineup Protocol** introduced in Buja, Cook, Hofmann, et al. (2009).

Embed a *target plot* (actual data) in a set of *null plots* (data generated under the null distribution).

].pull-right[

<img id='imglineupprotocol' src='images/lineup-protocol.png' alt=' ' width='100%'>

<div id='modallineupprotocol' class='modal'>
  
  <img class='modal-content' id='imgmodallineupprotocol'>
  
  <div id='captionlineupprotocol' class='modal-caption'></div>
</div>
]

???
Efforts in the field of graphics have developed graphical testing tools and methods such as the lineup protocol to provide a framework for inferential testing.

When inspecting a plot, how do we know if what we are seeing is actually there? One way of answering this question is to embed the true data plot (called target plot) into a set of randomly permuted data sets (called null plots). This is what we call a lineup.

This is similar to the law-enforcement procedure to line up a suspect among a set of innocents to check if a victim can identify the suspect as the perpetrator of the crime.

Here, visual evaluation of the lineup is conducted by a person. If the viewers detect the target plot, we can conclude the plots are distinguishable.

The lineup protocol is one such example of the development of tools designed for statistical graphical testing. The advancement of graphing software provides the tools necessary to develop new methods of testing graphics.

---
class:primary
# Eye Fitting Straight Lines
## Mosteller, Siegel, Trapido, et al. (1981)
.pull-left[
+ **Big Idea:** Students fitted lines by eye to four sets of points.
+ **Method:** 8.5 x 11 inch transparency with a straight line etched across the middle.
+ **Sample:** 153 graduate students and post docs in Introductory Biostatistics.
+ **Experimental Design:** Latin square.
+ **Findings:** Students tended to fit the slope of the first principal component.
].pull-right[
<img src="images/eyefitting-straight-lines-plots.png" width="95%"/>
]

???

I want to introduce a study conducted in 1981 called Eye Fitting Straight Lines by Mosteller et al. In this study:

+ Students fitted lines by eye to four sets of points.
+ 8.5 x 11 inch transparency with a straight line etched across the middle.
+ 153 graduate students and post docs in Introductory Biostatistics.
+ Latin square.
+ Students tended to fit the slope of the first principal component or major axis (the line that minimizes the sum of squares of perpendicular rather than vertical distances).

---
class:primary
# 'You Draw It' Feature 
## (New York Times, 2015)
.pull-left[
<img src="images/nyt-caraccidents-frame4.png" width="100%"/>
.center[
(Katz, 2017)
]
].pull-right[

Readers are asked to input their own assumptions about various metrics and compare how these assumptions relate to reality.

+ [Family Income affects college chances](https://www.nytimes.com/interactive/2015/05/28/upshot/you-draw-it-how-family-income-affects-childrens-college-chances.html) (Aisch, Cox, and Quealy, 2015)
+ [Just How Bad Is the Drug Overdose Epidemic?](https://www.nytimes.com/interactive/2017/04/14/upshot/drug-overdose-epidemic-you-draw-it.html) (Katz, 2017)
+ [What Got Better or Worse During Obama’s Presidency](https://www.nytimes.com/interactive/2017/01/15/us/politics/you-draw-obama-legacy.html?_r=0) (Buchanan, Park, and Pearce, 2017)
]

???

In 2015, the New York Times developed a You Draw it feature where readers are asked to input their own assumptions about various metrics and compare how these assumptions relate to reality.

The New York Times team utilizes **Data Driven Documents (D3)** that allows readers to predict these metrics through the use of drawing a line on their computer screen with their mouse.

---
class:primary
# Research Objectives

**What:**

1. Establish ‘You Draw It’, adapted from the New York Times feature, as
a tool for graphical testing.

2. Provide formal testing and support for the perception of linear regression.

**How:** Replicate the study conducted by Mosteller et al. (1981) using the ‘You Draw It’ method.

???

---
class:primary
# 'You Draw It' Task

Study Participant Prompt: *Use your mouse to fill in the trend in the yellow box region.*

.center[
<img src="images/eyefitting_example.gif" width="60%"/>
]

???

Here we see an example of a "You Draw It" task plot used in the study. Participants are prompted to "Use your mouse to fill in the trend in the yellow box region. The yellow box region moves along as the participant draws their trend-line until the yellow region disappears."

Task plots were created using Data Driven Documents (D3), a JavaScript-based graphing framework that facilitates user interaction. We then integrate this into RShiny using the r2d3 package.

---
class:primary
# Data Generation

.pull-left[

`\(N = 30\)` points `\((x_i, y_i), i = 1,...N\)` were generated for `\(x_i \in [x_{min}, x_{max}]\)`.

Data were simulated based on linear model with additive errors:
`\begin{equation}
y_i = \beta_0 + \beta_1 x_i + e_i
\end{equation}`

where `\(e_i \sim N(0, \sigma^2).\)`

Parameters `\(\beta_0\)` and `\(\beta_1\)` were selected to reflect the four data sets used in Mosteller, Siegel, Trapido, et al. (1981).

].pull-right[

<img id='imgeyefittingexamplesimplot' src='images/eyefitting-example-simplot.png' alt=' ' width='100%'>

<div id='modaleyefittingexamplesimplot' class='modal'>
  
  <img class='modal-content' id='imgmodaleyefittingexamplesimplot'>
  
  <div id='captioneyefittingexamplesimplot' class='modal-caption'></div>
</div>
]

???

Data were generated following a linear model with additive errors.

Model equation parameters, `\(\beta_0\)` and `\(\beta_1\)`, were selected to reflect the four data sets (F, N, S, and V) used in Mosteller et al. (1981).

+ **S:** positive slope; small variance; `\(x \in [0, 20]\)`. 
+ **F:** positive slope; a large variance; `\(x \in [0, 20]\)`.  
+ **V:** steep positive slope; small variance; `\(x \in [4, 16]\)`. 
+ **N:** negative slope; large variance; `\(x \in [0, 20]\)`.

---
class:primary
# Study Design

+ Participants recruited through Twitter, Reddit, and direct email in May 2021.

+ A total of 35 individuals completed 119 unique you draw it task plots.

+ Data sets were generated randomly, independently for each participant at the start of the experiment.

+ Participants shown 2 practice plots followed by 4 task plots randomly assigned for each individual in a completely randomized design.

+ Experiment conducted and distributed through an RShiny application found [**here**](https://emily-robinson.shinyapps.io/you-draw-it-pilot-app/).

???
Participants were recruited through through Twitter, Reddit, and direct email in May 2021. The experiment was conducted and distributed through an RShiny application. Participants were first shown 2 practice plots followed by the 4 You Draw It task plots randomly assigned for each individual in a completely randomized design.

---
class:primary
# Fitted Regression Lines

Compare participant drawn lines to:

+ Ordinary least squares regression line (OLS). 
+ Regression line based on principal axis (PCA).

.center[
<img src="images/pca-plot.jpg" width="80%"/>
]

???

We compare the participant drawn line to two regression lines determined by ordinary least squares regression and regression based on the principal axis. The figure illustrates the difference between an OLS regression line which minimizes the vertical distance of points from the line and a regression line based on the principal axis (Principal Component) which minimizes the Euclidean distance of points (orthogonal) from the line. This is what we refer to as “ensemble perception” indicating the visual system can compute averages of various features in parallel across the items in a set (in this case, over the x and y-axes).

---
class:primary
# Model Data

.pull-left[
For each participant, the final data set used for analysis contains:
+ `\(x_{ijk}\)`, `\(y_{ijk,drawn}\)`, `\(\hat y_{ijk,OLS}\)`, `\(\hat y_{ijk,PCA}\)`

for 
+ parameter choice `\(i = 1,2,3,4\)`, 
+ participant j = `\(1,...N_{participant}\)` 
+ `\(x_{ijk}\)` value corresponding to increment `\(k = 1, ...,4 x_{max} + 1\)`.

**Vertical residuals** between the drawn and fitted values were calculated as: 
+ `\(e_{ijk,OLS} = y_{ijk,drawn} - \hat y_{ijk,OLS}\)`
+ `\(e_{ijk,PCA} = y_{ijk,drawn} - \hat y_{ijk,PCA}\)`.

].pull-right[

<img id='imgeyefittingtrialplot' src='images/eyefitting-trial-plot.png' alt=' ' width='100%'>

<div id='modaleyefittingtrialplot' class='modal'>
  
  <img class='modal-content' id='imgmodaleyefittingtrialplot'>
  
  <div id='captioneyefittingtrialplot' class='modal-caption'></div>
</div>
]

---
class:primary
# Linear Trend Constraint
The **Linear Mixed Model** equation for each fit (OLS and PCA) residuals is given by:
`\begin{equation}
e_{ijk,fit} = \left[\gamma_0 + \alpha_i\right] + \left[\gamma_{1} x_{ijk} + \gamma_{2i} x_{ijk}\right] + p_{j} + \epsilon_{ijk}
\end{equation}`
where

+ `\(e_{ijk,fit}\)` is the residual between the drawn and fitted y-values for the `\(i^{th}\)` parameter choice, `\(j^{th}\)` participant, and `\(k^{th}\)` increment of x-value corresponding to either the OLS or PCA fit
+ `\(\gamma_0\)` is the overall intercept
+ `\(\alpha_i\)` is the effect of the `\(i^{th}\)` parameter choice (F, S, V, N) on the intercept
+ `\(\gamma_1\)` is the overall slope for `\(x\)`
+ `\(\gamma_{2i}\)` is the effect of the parameter choice on the slope
+ `\(x_{ijk}\)` is the x-value for the `\(i^{th}\)` parameter choice, `\(j^{th}\)` participant, and `\(k^{th}\)` increment
+ `\(p_{j} \sim N(0, \sigma^2_{participant})\)` is the random error due to the `\(j^{th}\)` participant's characteristics
+ `\(\epsilon_{ijk} \sim N(0, \sigma^2)\)` is the residual error.

???

Using the `lmer` function in the lme4 package, a linear mixed model (LMM) is fit separately to the OLS and PCA residuals, constraining the fit to a linear trend.

---
class:primary
# Linear Trend Constraint

.center[
<img src="images/eyefitting-lmer-plot.png" width="85%"/>
]

???
Results indicate the estimated trends of PCA residuals (orange) appear to align closer to the y = 0 horizontal (dashed) line than the OLS residuals (blue). In particular, this trend is more prominent in parameter choices with large variances (F and N). These results are consistent to those found in Mosteller
et al. (1981) indicating participants fit a trend-line closer to the estimated regression line with the slope of based on the first principal axis than the estimated OLS regression line.

---
class:primary
# Smoothing Spline Trend
The **Generalized Additive Mixed Model** equation for each fit (OLS and PCA) residuals is given by:
`\begin{equation}
e_{ijk,fit} = \alpha_i + s_{i}(x_{ijk}) + p_{j} + s_{j}(x_{ijk})
\end{equation}`
where

+ `\(e_{ijk,fit}\)` is the residual between the drawn and fitted y-values for the `\(i^{th}\)` parameter choice, `\(j^{th}\)` participant, and `\(k^{th}\)` increment of x-value corresponding to either the OLS or PCA fit
+ `\(\alpha_i\)` is the intercept for the parameter choice `\(i\)`
+ `\(s_{i}\)` is the smoothing spline for the `\(i^{th}\)` parameter choice
+ `\(x_{ijk}\)` is the x-value for the `\(i^{th}\)` parameter choice, `\(j^{th}\)` participant, and `\(k^{th}\)` increment
+ `\(p_{j} \sim N(0, \sigma^2_{participant})\)` is the error due to participant variation
+ `\(s_{j}\)` is the random smoothing spline for each participant.

???

Eliminating the linear trend constraint, the `bam` function in the mgcv package is used to fit a generalized additive mixed model (GAMM) separately to the OLS and PCA residuals to allow for estimation of smoothing splines.

---
class:primary
# Smoothing Spline Trend

.center[
<img src="images/eyefitting-gamm-plot.png" width="85%"/>
]

???
The results of the GAMM align with those in the linear constraint trend providing support that for scatter-plots with more noise (F and N), estimated trends of PCA residuals (orange) appear to align closer to the y = 0 horizontal (dashed) line than the OLS residuals (blue). However, By fitting smoothing splines, we can determine whether participants naturally fit a straight trend-line to the set of points or whether they deviate throughout the domain providing us with further insight into the curvature humans perceive in a set of points.

---
class:primary
# Conclusion

**Research Objectives:**

1. Establish 'You Draw It' as a tool for testing graphics.
2. Provide experimental support for “ensemble perception”.

**Results:**
+ Estimated drawn trend-lines followed closer to the regression line based on the principal axes than the OLS regression line.
+ Most prominent in data simulated with large variances.
+ Humans perform “ensemble perception” in a statistical graphic setting.

**The reproducibility of these results serve as validation of the 'You Draw It' tool and method.**

???

1. Establish 'You Draw It' as a tool for testing graphics.
2. Provide experimental support for human perception of linear regression.

**Results:**
+ Estimated drawn trend-lines followed closer to the principal axes than the OLS regression line.
+ Most prominent in data simulated with large variances.
+ Humans perform “ensemble perception” in a statistical graphic setting as participants minimized the distance from the their regression line over both the x and y axis simultaneously

**The reproducibility of these results serve as validation of the 'You Draw It' tool and method.**

---
class:primary
# Future Work

.center[
<img src="images/loading.gif" width="50%"/>
]

✏️ Implement the 'You Draw It' method in non-linear settings.

📈 Evaluate human ability to extrapolate data from trends.

<br>

.right-col[
Gif Source: [photobucket.com](http://s280.photobucket.com/user/ariffisariff/media/animated-loading.gif.html)
]
---
class:primary
# References
<font size="2">
<p><cite><a id='bib-Aisch_NYTimes_presidential_forecast'></a><a href="#cite-Aisch_NYTimes_presidential_forecast">Aisch, G., N. Cohn, A. Cox, et al.</a>
(2016).
<em>Live Presidential Forecast</em>.
URL: <a href="https://www.nytimes.com/elections/2016/forecast/president">https://www.nytimes.com/elections/2016/forecast/president</a>.</cite></p>

<p><cite><a id='bib-aisch_cox_quealy_2015'></a><a href="#cite-aisch_cox_quealy_2015">Aisch, G., A. Cox, and K. Quealy</a>
(2015).
<em>You Draw It: How Family Income Predicts Children's College Chances</em>.
URL: <a href="https://www.nytimes.com/interactive/2015/05/28/upshot/you-draw-it-how-family-income-affects-childrens-college-chances.html">https://www.nytimes.com/interactive/2015/05/28/upshot/you-draw-it-how-family-income-affects-childrens-college-chances.html</a>.</cite></p>

<p><cite><a id='bib-buchanan_park_pearce_2017'></a><a href="#cite-buchanan_park_pearce_2017">Buchanan, L., H. Park, and A. Pearce</a>
(2017).
<em>You Draw It: What Got Better or Worse During Obama's Presidency</em>.
URL: <a href="https://www.nytimes.com/interactive/2017/01/15/us/politics/you-draw-obama-legacy.html">https://www.nytimes.com/interactive/2017/01/15/us/politics/you-draw-obama-legacy.html</a>.</cite></p>

<p><cite><a id='bib-buja2009statistical'></a><a href="#cite-buja2009statistical">Buja, A., D. Cook, H. Hofmann, et al.</a>
(2009).
&ldquo;Statistical inference for exploratory data analysis and model diagnostics&rdquo;.
In: <em>Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences</em> 367.1906, pp. 4361&ndash;4383.</cite></p>

<p><cite>Carpenter, P. A. and P. Shah
(1998).
&ldquo;A model of the perceptual and conceptual processes in graph comprehension.&rdquo;
In: <em>Journal of Experimental Psychology: Applied</em> 4.2, p. 75.</cite></p>

<p><cite><a id='bib-chandar2012graph'></a><a href="#cite-chandar2012graph">Chandar, N., D. Collier, and P. Miranti</a>
(2012).
&ldquo;Graph standardization and management accounting at AT&amp;T during the 1920s&rdquo;.
In: <em>Accounting History</em> 17.1, pp. 35&ndash;62.</cite></p>

<p><cite>Chong, S. C. and A. Treisman
(2003).
&ldquo;Representation of statistical properties&rdquo;.
In: <em>Vision research</em> 43.4, pp. 393&ndash;404.</cite></p>

<p><cite>&mdash;
(2005).
&ldquo;Statistical processing: Computing the average size in perceptual groups&rdquo;.
In: <em>Vision research</em> 45.7, pp. 891&ndash;900.</cite></p>

<p><cite><a id='bib-ciccione2021can'></a><a href="#cite-ciccione2021can">Ciccione, L. and S. Dehaene</a>
(2021).
&ldquo;Can humans perform mental regression on a graph? Accuracy and bias in the perception of scatterplots&rdquo;.
In: <em>Cognitive Psychology</em> 128, p. 101406.</cite></p>

<p><cite>Cleveland, W. S. and R. McGill
(1984).
&ldquo;Graphical perception: Theory, experimentation, and application to the development of graphical methods&rdquo;.
In: <em>Journal of the American statistical association</em> 79.387, pp. 531&ndash;554.</cite></p>
</font>

---
class:primary
# References
<font size="2">
<p><cite>Cleveland, W. S. and R. McGill
(1985).
&ldquo;Graphical perception and graphical methods for analyzing scientific data&rdquo;.
In: <em>Science</em> 229.4716, pp. 828&ndash;833.</cite></p>

<p><cite><a id='bib-finney1951subjective'></a><a href="#cite-finney1951subjective">Finney, D.</a>
(1951).
&ldquo;Subjective judgment in statistical analysis: An experimental study&rdquo;.
In: <em>Journal of the Royal Statistical Society: Series B (Methodological)</em> 13.2, pp. 284&ndash;297.</cite></p>

<p><cite><a id='bib-gouretski2007much'></a><a href="#cite-gouretski2007much">Gouretski, V. and K. P. Koltermann</a>
(2007).
&ldquo;How much is the ocean really warming?&rdquo;
In: <em>Geophysical Research Letters</em> 34.1.</cite></p>

<p><cite>Green, T. M. and B. Fisher
(2009).
&ldquo;The personal equation of complex individual cognition during visual interface interaction&rdquo;.
In: 
<em>Workshop on Human-Computer Interaction and Visualization</em>.
Springer.
, pp. 38&ndash;57.</cite></p>

<p><cite><a id='bib-harms1991august'></a><a href="#cite-harms1991august">Harms, H.</a>
(1991).
&ldquo;August Friedrich Wilhelm Crome (1753-1833) Autor begehrter Wirtschaftskarten&rdquo;.
In: <em>Cartographica Helvetica</em> 3, pp. 33&ndash;38.</cite></p>

<p><cite>Hofmann, H., L. Follett, M. Majumder, et al.
(2012).
&ldquo;Graphical tests for power comparison of competing designs&rdquo;.
In: <em>IEEE Transactions on Visualization and Computer Graphics</em> 18.12, pp. 2441&ndash;2448.</cite></p>

<p><cite><a id='bib-katz_2017'></a><a href="#cite-katz_2017">Katz, J.</a>
(2017).
<em>You Draw It: Just How Bad Is the Drug Overdose Epidemic?</em>
URL: <a href="https://www.nytimes.com/interactive/2017/04/14/upshot/drug-overdose-epidemic-you-draw-it.html">https://www.nytimes.com/interactive/2017/04/14/upshot/drug-overdose-epidemic-you-draw-it.html</a>.</cite></p>

<p><cite><a id='bib-lewandowsky1989perception'></a><a href="#cite-lewandowsky1989perception">Lewandowsky, S. and I. Spence</a>
(1989).
&ldquo;The perception of statistical graphs&rdquo;.
In: <em>Sociological Methods &amp; Research</em> 18.2-3, pp. 200&ndash;242.</cite></p>

<p><cite><a id='bib-mosteller1981eye'></a><a href="#cite-mosteller1981eye">Mosteller, F., A. F. Siegel, E. Trapido, et al.</a>
(1981).
&ldquo;Eye fitting straight lines&rdquo;.
In: <em>The American Statistician</em> 35.3, pp. 150&ndash;152.</cite></p>

<p><cite><a id='bib-playfair1801statistical'></a><a href="#cite-playfair1801statistical">Playfair, W.</a>
(1801).
&ldquo;The statistical breviary; shewing, on a principle entirely new, the resources of every state and kingdom in Europe, Wallis, Londres&rdquo;.
In: <em>Press, Chicago</em>.</cite></p>
</font>

---
class:primary
# References
<font size="2">
<p><cite>Spence, I.
(1990).
&ldquo;Visual psychophysics of simple graphical elements.&rdquo;
In: <em>Journal of Experimental Psychology: Human Perception and Performance</em> 16.4, p. 683.</cite></p>

<p><cite>Unwin, A.
(2020).
&ldquo;Why is data visualization important? what is important in data visualization?&rdquo;
In: <em>Harvard Data Science Review</em> 2.1.</cite></p>

<p><cite>Van Opstal, F., F. P. de
Lange, and S. Dehaene
(2011).
&ldquo;Rapid parallel semantic processing of numbers without awareness&rdquo;.
In: <em>Cognition</em> 120.1, pp. 136&ndash;147.</cite></p>

<p><cite>Vanderplas, S., D. Cook, and H. Hofmann
(2020).
&ldquo;Testing Statistical Charts: What makes a good graph?&rdquo;
In: <em>Annual Review of Statistics and Its Application</em> 7, pp. 61&ndash;88.</cite></p>

<p><cite>VanderPlas, S. and H. Hofmann
(2015).
&ldquo;Spatial reasoning and data displays&rdquo;.
In: <em>IEEE Transactions on Visualization and Computer Graphics</em> 22.1, pp. 459&ndash;468.</cite></p>

<p><cite>&mdash;
(2017).
&ldquo;Clusters beat trend!? testing feature hierarchy in statistical graphics&rdquo;.
In: <em>Journal of Computational and Graphical Statistics</em> 26.2, pp. 231&ndash;242.</cite></p>

<p><cite><a id='bib-walker2013statistical'></a><a href="#cite-walker2013statistical">Walker, F. A.</a>
(2013).
<em>Statistical atlas of the United States based on the results of the ninth census 1870 with contributions from many eminent men of science and several departments of the government</em>.</cite></p>

<p><cite>Wickham, H.
(2011).
&ldquo;ggplot2&rdquo;.
In: <em>Wiley Interdisciplinary Reviews: Computational Statistics</em> 3.2, pp. 180&ndash;185.</cite></p>

<p><cite>Wilkinson, L.
(2013).
<em>The grammar of graphics</em>.
Springer Science &amp; Business Media.</cite></p>

<p><cite><a id='bib-yates1985graphs'></a><a href="#cite-yates1985graphs">Yates, J.</a>
(1985).
&ldquo;Graphs as a managerial tool: A case study of Du Pont's use of graphs in the early twentieth century&rdquo;.
In: <em>The Journal of Business Communication (1973)</em> 22.1, pp. 5&ndash;33.</cite></p>
</font>

---
class:inverse
<br>
<br>
<br>
<br>
<br>
<br>
.center[
# Thank you!
<br
<br>
<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:white;overflow:visible;position:relative;"><path d="M502.3 190.8c3.9-3.1 9.7-.2 9.7 4.7V400c0 26.5-21.5 48-48 48H48c-26.5 0-48-21.5-48-48V195.6c0-5 5.7-7.8 9.7-4.7 22.4 17.4 52.1 39.5 154.1 113.6 21.1 15.4 56.7 47.8 92.2 47.6 35.7.3 72-32.8 92.3-47.6 102-74.1 131.6-96.3 154-113.7zM256 320c23.2.4 56.6-29.2 73.4-41.4 132.7-96.3 142.8-104.7 173.4-128.7 5.8-4.5 9.2-11.5 9.2-18.9v-19c0-26.5-21.5-48-48-48H48C21.5 64 0 85.5 0 112v19c0 7.4 3.4 14.3 9.2 18.9 30.6 23.9 40.7 32.4 173.4 128.7 16.8 12.2 50.2 41.8 73.4 41.4z"/></svg> **emily.robinson@huskers.unl.edu**
<svg aria-hidden="true" role="img" viewBox="0 0 576 512" style="height:1em;width:1.12em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:white;overflow:visible;position:relative;"><path d="M280.37 148.26L96 300.11V464a16 16 0 0 0 16 16l112.06-.29a16 16 0 0 0 15.92-16V368a16 16 0 0 1 16-16h64a16 16 0 0 1 16 16v95.64a16 16 0 0 0 16 16.05L464 480a16 16 0 0 0 16-16V300L295.67 148.26a12.19 12.19 0 0 0-15.3 0zM571.6 251.47L488 182.56V44.05a12 12 0 0 0-12-12h-56a12 12 0 0 0-12 12v72.61L318.47 43a48 48 0 0 0-61 0L4.34 251.47a12 12 0 0 0-1.6 16.9l25.5 31A12 12 0 0 0 45.15 301l235.22-193.74a12.19 12.19 0 0 1 15.3 0L530.9 301a12 12 0 0 0 16.9-1.6l25.5-31a12 12 0 0 0-1.7-16.93z"/></svg> **www.emilyarobinson.com**
<svg aria-hidden="true" role="img" viewBox="0 0 496 512" style="height:1em;width:0.97em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:white;overflow:visible;position:relative;"><path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"/></svg> **earobinson95**
]