‘You Draw It’: Implementing human interaction in R shiny with r2d3

class:title-slide-custom

<style>

/* colors: #EEB422, #8B0000, #191970, #00a8cc */
/* define the new color palette here! */
a, a > code {
  color: #8B0000;
  text-decoration: none;
}

.title-slide h2::after, .mline h1::after {
  content: '';
  display: block;
  border: none;
  background-color: #8B0000;
  color: #8B0000;
  height: 2px;
}

.remark-slide-content {
  background-color: #FFFFFF;
  border-top: 80px solid #8B0000;
  font-size: 20px;
  font-weight: 300;
  line-height: 1.5;
  
  background-image: url(css/UNL.svg);
	background-position: 2% 98%;
	background-size: 10%;
	border-bottom: 0;
}

.inverse {
  background-color: #8B0000;
  
  
	
  
}

.remark-slide-content > h1 {
  font-family: 'Roboto';
  font-weight: 300;
  font-size: 45px;
  margin-top: -95px;
  margin-left: -00px;
  color: #FFFFFF;
}

.title-slide {
  background-color: #FFFFFF;
  
  background-image:  url(css/UNL.svg);
	background-position: 98% 98%;
	
	background-size: 20%;
	border-bottom: 0;
	border: 10px solid 	#8B0000;
	
}

.title-slide > h1  {
  color: #111111;
  font-size: 32px;
  text-shadow: none;
  font-weight: 500;
  text-align: left;
  margin-left: 15px;
  padding-top: 80px;
}
.title-slide > h2  {
  margin-top: -25px;
  padding-bottom: -20px;
  color: #111111;
  text-shadow: none;
  font-weight: 100;
  font-size: 28px;
  text-align: left;
  margin-left: 15px;
}
.title-slide > h3  {
  color: #111111;
  text-shadow: none;
  font-weight: 100;
  font-size: 28px;
  text-align: left;
  margin-left: 15px;
  margin-bottom: -20px;
}

body {
  font-family: 'Roboto';
  font-weight: 300;
}

.remark-slide-number {
  font-size: 13pt;
  font-family: 'Roboto';
  color: #272822;
  opacity: 1;
}

.inverse .remark-slide-number {
  font-size: 13pt;
  font-family: 'Roboto';
  color: #FAFAFA;
  opacity: 1;
}

.title-slide-custom .remark-slide-number {
  display: none;
}

.title-slide-custom h3::after, .mline h1::after {
  content: '';
  display: block;
  border: none;
  background-color: #8B0000;
  color: #8B0000;
  height: 2px;
}

.title-slide-custom {
  background-color: #FFFFFF;
  
  background-image:  url(css/UNL.svg);
	background-position: 98% 98%;
	
	background-size: 20%;
	border-bottom: 0;
	border: 10px solid 	#8B0000;
	
}

.title-slide-custom > h1  {
  color: #111111;
  font-size: 40px;
  text-shadow: none;
  font-weight: 500;
  text-align: left;
  margin-left: 15px;
  padding-top: 80px;
  padding-bottom: 10px;
}
.title-slide-custom > h2  {
  margin-top: -25px;
  padding-bottom: 30px;
  color: #111111;
  text-shadow: none;
  font-weight: 100;
  font-size: 32px;
  text-align: left;
  margin-left: 15px;
}
.title-slide-custom > h3  {
  margin-top: -25px;
  padding-bottom: -25px;
  color: #111111;
  text-shadow: none;
  font-weight: 100;
  font-size: 32px;
  text-align: left;
  margin-left: 15px;
}
.title-slide-custom > h4  {
  color: #111111;
  text-shadow: none;
  font-weight: 100;
  font-size: 28px;
  text-align: left;
  margin-left: 15px;
  margin-bottom: -30px;
  padding-bottom: -25px;
}
.title-slide-custom > h5  {
  color: #111111;
  text-shadow: none;
  font-weight: 100;
  font-size: 24px;
  text-align: left;
  margin-left: 15px;
  margin-bottom: -40px;
}

</style>

<br><br><br>
## ‘You Draw It’: Implementing human interaction in R shiny with r2d3
### 2022 SDSS
#### Emily A. Robinson
#### Department of Statistics, University of Nebraska - Lincoln
##### <svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:black;overflow:visible;position:relative;"><path d="M502.3 190.8c3.9-3.1 9.7-.2 9.7 4.7V400c0 26.5-21.5 48-48 48H48c-26.5 0-48-21.5-48-48V195.6c0-5 5.7-7.8 9.7-4.7 22.4 17.4 52.1 39.5 154.1 113.6 21.1 15.4 56.7 47.8 92.2 47.6 35.7.3 72-32.8 92.3-47.6 102-74.1 131.6-96.3 154-113.7zM256 320c23.2.4 56.6-29.2 73.4-41.4 132.7-96.3 142.8-104.7 173.4-128.7 5.8-4.5 9.2-11.5 9.2-18.9v-19c0-26.5-21.5-48-48-48H48C21.5 64 0 85.5 0 112v19c0 7.4 3.4 14.3 9.2 18.9 30.6 23.9 40.7 32.4 173.4 128.7 16.8 12.2 50.2 41.8 73.4 41.4z"/></svg> [emily.robinson@huskers.unl.edu](emily.robinson@huskers.unl.edu)
##### <svg aria-hidden="true" role="img" viewBox="0 0 576 512" style="height:1em;width:1.12em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:black;overflow:visible;position:relative;"><path d="M280.37 148.26L96 300.11V464a16 16 0 0 0 16 16l112.06-.29a16 16 0 0 0 15.92-16V368a16 16 0 0 1 16-16h64a16 16 0 0 1 16 16v95.64a16 16 0 0 0 16 16.05L464 480a16 16 0 0 0 16-16V300L295.67 148.26a12.19 12.19 0 0 0-15.3 0zM571.6 251.47L488 182.56V44.05a12 12 0 0 0-12-12h-56a12 12 0 0 0-12 12v72.61L318.47 43a48 48 0 0 0-61 0L4.34 251.47a12 12 0 0 0-1.6 16.9l25.5 31A12 12 0 0 0 45.15 301l235.22-193.74a12.19 12.19 0 0 1 15.3 0L530.9 301a12 12 0 0 0 16.9-1.6l25.5-31a12 12 0 0 0-1.7-16.93z"/></svg> [www.emilyarobinson.com](https://www.emilyarobinson.com/)
##### <svg aria-hidden="true" role="img" viewBox="0 0 496 512" style="height:1em;width:0.97em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:black;overflow:visible;position:relative;"><path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"/></svg> [earobinson95](https://github.com/earobinson95)
<br><br>
.medium[*Slides: https://bit.ly/3xav05Y*]

???

Thank you everyone for coming, it's great to see you at this session. Today, I am going to be sharing some of my work on using R2D3 and Shiny to test statistical graphics.

---
class:primary
# Outline

<i class="fa fa-chart-line" role="presentation" aria-label="chart-line icon"></i> Background of graphical testing and measuring patterns

<i class="fab fa-js" role="presentation" aria-label="js icon"></i> <i class="fa fa-plus" role="presentation" aria-label="plus icon"></i> <i class="fab fa-r-project" role="presentation" aria-label="r-project icon"></i> Integration of D3.js and R Shiny

<i class="fa fa-pen" role="presentation" aria-label="pen icon"></i> Test your drawing skills!

<i class="fa fa-check" role="presentation" aria-label="check icon"></i> Validation and Application

<i class="fa fa-spinner" role="presentation" aria-label="spinner icon"></i> Future Work

???

During my talk, I will first provide a brief background on graphical testing and how it relates to detecting trends and patterns in data. Then I will talk about the development process behind the scenes and you will get a chance to test out your drawing skills. Finally, I will share results from the validation study and plans for future work.

---
class:primary
# Background

**Testing statistical graphics**

+ Evaluate design choices and understand cognitive biases through the use of visual tests.
+ Researchers conduct studies in which human subjects are asked to conduct tasks related to the perception of statistical charts such as differentiation, prediction, estimation, and extrapolation.

???

We all use statistical graphics, but how do we know that the graphics we use are communicating properly? Through experimentation, graphical testing methods allow researchers to conduct studies geared at understanding human ability to conduct tasks related to the perception of statistical charts such as differentiation, prediction, estimation, and extrapolation. Efforts in the field of statistical graphics have developed graphical testing tools and methods and all of these types of tests require different levels of use and manipulation of the information being presented in the chart. With the advancement of graphing software, we have the tools necessary to develop new methods of testing.

**Measuring Patterns & Trends**

+ Our visual system is naturally built to look for structure and identify patterns.
+ Physical manipulation: maneuvering a string, black thread, or ruler until the fit is suitable, then drawing the line through the set of points (Mosteller, Siegel, Trapido, et al., 1981; Finney, 1951)
+ Ciccione and Dehaene (2021) conducted a comprehensive set of studies based on psychophysical approaches.

???

One such aspect of interest is the ability to identify and detect trends. Our visual system is naturally built to look for structure and identify patterns. For instance, points going down from left to right indicates a negative correlation between the x and y variables. Initial studies in the 20th century explored the use of fitting lines by eye through a set of points through physical manipulation methods such as maneuvering a string, black thread, or ruler until the fit is suitable, then drawing the line through the set of points. Recently, a study in 2021 conducted a comprehensive set of studies investigating human ability to detect trends in graphical representations from a psychophysical approach.

**How can we compare our intuitive visual sense of patterns to those determined by statistical methods?**

???

**How can we compare our intuitive visual sense of patterns to those determined by statistical methods?**

---
class:primary
# 'You Draw It' feature 
## (New York Times, 2015)
.pull-left[
<img src="images/nyt-caraccidents.png" width="100%"/>
.center[
(Katz, 2017)
]
].pull-right[

Readers are asked to input their own assumptions about various metrics and compare how these assumptions relate to reality.

+ [Family Income affects college chances](https://www.nytimes.com/interactive/2015/05/28/upshot/you-draw-it-how-family-income-affects-childrens-college-chances.html) (Aisch, Cox, and Quealy, 2015)
+ [Just How Bad Is the Drug Overdose Epidemic?](https://www.nytimes.com/interactive/2017/04/14/upshot/drug-overdose-epidemic-you-draw-it.html) (Katz, 2017)
+ [What Got Better or Worse During Obama’s Presidency](https://www.nytimes.com/interactive/2017/01/15/us/politics/you-draw-obama-legacy.html?_r=0) (Buchanan, Park, and Pearce, 2017)
]

???

In 2015, the New York Times introduced an interactive feature, called ‘You Draw It’, where readers input their own assumptions about various metrics and compare how these assumptions relate to reality. The Times team utilizes Data Driven Documents (D3) that allows readers to predict these metrics through the use of drawing a line on their computer screen with their mouse.

The New York Times team utilizes **Data Driven Documents (D3)** that allows readers to predict these metrics through the use of drawing a line on their computer screen with their mouse.

???

The goal of this research is to implement ‘You Draw It,’ adapted from the New York Times feature, as a way to measure the patterns we see in data. In the rest of the presentation, I am going to provide a high level overview of the technical details of the software development which utilized interactive graphics in R. I will then share results from a study which validates ‘You Draw It’ as a method for graphical testing and recommend an appropriate data analysis method to the participant data.

---
class:primary
# Background of D3

**Who?** [Mike Bostock](https://observablehq.com/@mbostock) created D3 during his time working on graphics at the New York Times.
  
**What?** Open-source JavaScript based graphing framework
+ D3 = "Data Driven Documents"
+ `D3` is to JavaScript as `ggplot2` is to R
+ Framework for binding objects and layers to plotting area
  + framework for movement and user interaction

**When?** D3 v1.0 released in 2011.

**Where?** The internet!

**Why?** Advantages of using D3 include animation and allowing for movement and user interaction.

**How?** `r2d3`!

<br>
.pull-right[.pull-right[
.medium[*Modified from* [*Kiegan Rice.*](https://kiegan.github.io/talks/graphics-group-r2d3/graphics-group-r2d3.html#1)
]]]
???

Data Driven Documents (D3) is a JavaScript-based graphing framework that facilitates user interaction. It is used by major news and research organizations such as the New York Times, FiveThirtyEight, Washington Post, and the Pew Research Center to create and customize graphics.

---
class:primary
# Relationship between D3 and R

.pull-left[
The `r2d3` package (Strayer, Luraschi, and Allaire, 2020) in R provides an efficient integration of D3 visuals and R by displaying them in familiar formats:
+ RMarkdown with HTML output
+ Shiny applications (amazing!)

].pull-right[
.center[ 
 <img src="images/r2d3-hex.png" width="40%"/>
]
]

`r2d3` makes it easy to do your data processing in R, then apply D3.js code to visualize that data!

???

A challenge of working with D3 is the environment necessary to display the graphics and images.

The `r2d3` package in R provides an efficient integration of D3 visuals and R by displaying them in familiar formats:
+ RMarkdown with HTML output
+ Shiny applications (amazing!)

`r2d3` makes it easy to do your data processing in R, then apply D3.js code to visualize that data!

--
.pull-left[
**How?**
+ Converts data in R to JSON that can be interpreted by JavaScript
+ Sources D3 code library
+ Creates plot container (svg)
+ Renders plot using source code
].right-plot[

```r
r2d3(data = data,
     script = "d3-source-code.js",
     d3_version= "5")
```
]

<br><br><br>
.pull-right[.pull-right[
.medium[*Modified from* [*Kiegan Rice.*](https://kiegan.github.io/talks/graphics-group-r2d3/graphics-group-r2d3.html#1)
]]]

???

The example R code illustrates the structure of the r2d3 function which includes specification of a data frame in R (converted to a JSON file), the D3.js source code file, and the D3 version that accompanies the source code. The r2d3 function then renders the plot using the source code.

---
class:primary
# Getting started with D3

`D3.js` is to JavaScript as `ggplot2` is to R

.pull-left[
<i class="fab fa-js" role="presentation" aria-label="js icon"></i> [Codecademy: Introduction to JavaScript](https://www.codecademy.com/learn/introduction-to-javascript)

<i class="fa fa-image" role="presentation" aria-label="image icon"></i> Understand [SVG](http://tutorials.jenkov.com/svg/g-element.html) elements: inspect elements in web browser!

<i class="fa fa-book-open" role="presentation" aria-label="book-open icon"></i> Amelia Wattenberger's [Full Stack D3 and Data Visualization Book](https://www.newline.co/fullstack-d3)

<i class="fa fa-chart-bar" role="presentation" aria-label="chart-bar icon"></i> Build a basic graphic using [r2d3](https://rstudio.github.io/r2d3/articles/introduction.html)

<i class="fa fa-grimace" role="presentation" aria-label="grimace icon"></i> Modify `D3.js` code until it does what you want!
]
.pull-right[
**Additional Resources**

<i class="fa fa-code" role="presentation" aria-label="code icon"></i> [How to learn D3 with no coding experience](https://www.heshameissa.com/blog/learn-d3)

<i class="fab fa-twitter" role="presentation" aria-label="twitter icon"></i> Amelia Wattenberger on [Twitter](https://twitter.com/Wattenberger)
]

???

When I first started working on this, I was comfortable working with R and Shiny, however, I had almost no experience working with JavaScript. I wanted to share some resources that I found useful in learning to create graphics using D3. One thing to know is that D3 uses SVG's (scalable vector graphics) which preserves shapes rather than pixels and follows a layered element framework. I highly recommend Amelia Wattenberger's Full Stack D3 and Data Visualization book. She does not use R html formats as her environment, but the examples and D3 source code can easily be implemented in any formats you might be familiar with.

---
class:primary
# 'You Draw It' task plot

Prompt: *Use your mouse to fill in the trend in the yellow box region.*

.pull-left[
<img src="images/eyefitting_example.gif" width="100%"/>
].pull-right[
<img src="images/can-you-draw-it-mobile-qrcode.png" width="85%"/>
]

**Test out your drawing skills by scanning the QR code on your mobile device or visit [emily-robinson.shinyapps.io/can-you-draw-it](https://emily-robinson.shinyapps.io/can-you-draw-it) on a laptop.**

???

Here you see an example of 'You Draw It' interactive plots. In graphical testing, participants are prompted to: "Use your mouse to fill in the trend in the yellow box region". The yellow box region then moves along as the participant draws their trend-line until the yellow region disappears.

I am going to give everyone a few minutes to scan the QR code on your mobile device or tablet to test out your drawing skills.

If you have a laptop, you can visit the link on the slide to take a look at the working pieces behind the scenes with showcase display mode. We will talk a little more about this on the next slides as well.

---
class:primary
# Code sketch

<br>
.center[ 
 
 <img src="images/code-sketch-2.png" width="100%"/>
]

???

We conduct all data simulation and processing in R and output two data sets - point data and line data -
containing (x, y) coordinates corresponding to either a simulated point or fitted value predicted by a statistical model respectively. Then, the r2d3 package converts the data sets in R to JSON to be interpreted by the D3.js code. We define functions in D3.js to draw the initial plot and set up drawable points for the user drawn line. Drag events in D3.js are utilized to observe and react to user input. Shiny Messages are used to communicate the user interaction between the D3 code and the R environment. The plot is then rendered and updated on user interaction into the R shiny application with the `RenderD3` and `d3Output` functions. Once the user is done drawing the line, we can save the results of the drawn line to a database for analysis.

---
class:primary
# Challenges

.pull-left[
+ Converting between pixels and coordinate points.

+ Passing two data sets into `r2d3()` requires converting to json first.

+ Layering elements with opacity.

+ D3 version matters!

+ Only works with a one-to-one function.
]
.pull-right[
 <img src="images/d3-v5-commit-frustrations-2.png" width="100%"/>
]

???

During the development process, there were a few challenges we had to overcome. Afer looking back at my GitHub commits, I outlined a few places where I found frustrations.

+ One thing to know when working with D3 is that because it uses SVG's (scalble vector graphics), you need to convert between pixels and coordinates.

+ On the slide previously, I talked about how we had "point data" and "line data" - this was a challenge to read in both data sets and be able to reference them within the JavaScript code.

+ I had challenges trying to get the layers in the right order so that the yellow box region would show up under the points while still allowing the grid lines to appear.

+ At one point during the process I updated my R and packages - this in turn updated my D3 version and I really thought I broke everything for a few days.

+ Something we are still thinking through for the future is that this only works with a one-to-one function and does not allow multiple estimates or predictions for one x-value.

---
class:primary
# Validation study

Replicated *Eye Fitting Straight Lines* by Mosteller, Siegel, Trapido, et al. (1981)

.pull-left[
+ **Big Idea:** Students fitted lines by eye to four sets of points.
+ **Method:** 8.5 x 11 inch transparency with a straight line etched across the middle.
+ **Sample:** 153 graduate students and post docs in Introductory Biostatistics.
+ **Experimental Design:** Latin square.
+ **Findings:** Students tended to fit the slope of the first principal component.
].pull-right[ 
<img src="images/eyefitting-straight-lines-plots.png" width="75%"/>
]

???

We conducted a study in order to validate ‘You Draw It’ as a method for graphical testing, comparing results
to the less technological method utilized in Mosteller et al. (1981). In this study, participants were asked to fit lines by eye to four sets of points using an 8.5 x 11 inch transparency. Their results showed that students tended to fit the slope of the first principal component as opposed to the slope of the ordinary least squares regression.

Experiment conducted and distributed through an R Shiny application found at [emily-robinson.shinyapps.io/you-draw-it-validation-applet/](https://emily-robinson.shinyapps.io/you-draw-it-validation-applet/) in May 2021 with 35 participants completing 119 unique 'You Draw It' task plots.

???

In our study, participants were asked to draw a line through a set of points with the 'You Draw It' method. We simulated data meant to reflect the four data sets in the 1981 study. This experiment was conducted and distributed through an R Shiny application.

---
class:primary
# Validation study
 
.pull-left[

Compare participant drawn line to **statistical regression results**.

<img id='imgpcaplot' src='images/pca-plot.jpg' alt=' ' width='100%'>

<div id='modalpcaplot' class='modal'>
  
  <img class='modal-content' id='imgmodalpcaplot'>
  
  <div id='captionpcaplot' class='modal-caption'></div>
</div>

].pull-right[

Interested in the **vertical residuals** between the drawn and fitted values.

<img id='imgeyefittingtrialplot' src='images/eyefitting-trial-plot.png' alt=' ' width='100%'>

<div id='modaleyefittingtrialplot' class='modal'>
  
  <img class='modal-content' id='imgmodaleyefittingtrialplot'>
  
  <div id='captioneyefittingtrialplot' class='modal-caption'></div>
</div>
]

???
We compare the participant drawn line to two regression lines determined by ordinary least squares regression and regression based on the principal axis. This figure illustrates the difference between an OLS regression line which minimizes the vertical distance of points from the line and a regression line based on the principal axis (Principal Component) which minimizes the Euclidean distance of points (orthogonal) from the line, capturing the variability in both the horizontal and vertical directions.

From our study, we collect data on the participant drawn values, the fitted values from the ordinary least squares regression, and the fitted values from the regression based on the principal axis for that unique data set. We are mainly interested in the deviation of the participant drawn line from the fitted regression lines. So while it seems counter-intuitive, the residual actually becomes our response in this case.

---
class:primary
# Validation study

.pull-left[
**Linear Trend Constraint**

.center[

<img id='imgeyefittinglmerplot' src='images/eyefitting-lmer-plot.png' alt=' ' width='110%'>

<div id='modaleyefittinglmerplot' class='modal'>
  
  <img class='modal-content' id='imgmodaleyefittinglmerplot'>
  
  <div id='captioneyefittinglmerplot' class='modal-caption'></div>
</div>
]
].pull-right[

**Generalized Additive Mixed Model (GAMM)**

.center[

<img id='imgeyefittinggammplot' src='images/eyefitting-gamm-plot.png' alt=' ' width='110%'>

<div id='modaleyefittinggammplot' class='modal'>
  
  <img class='modal-content' id='imgmodaleyefittinggammplot'>
  
  <div id='captioneyefittinggammplot' class='modal-caption'></div>
</div>
]
]

???

Linear mixed model and a Generalized Additive Mixed Model (GAMM) were used to evaluate the vertical residuals in order to statistically compare visually fitted trends to actual metrics, simulated data models, or statistical regression results. A benefit of using a GAMM is the estimation of smoothing splines, allowing for flexibility in the residual trend.

Results from our study were consistent with those found in the previous study; when shown points
following a linear trend, participants tended to fit the slope of the first principal component over the slope of the least-squares regression line. This trend was most prominent when shown data simulated with larger variances. This study reinforces the differences between intuitive visual model fitting and statistical model fitting, providing information about human perception as it relates to the use of statistical graphics. This also laid the foundation for recommending the use of GAMMS as an analysis method of the collected data.

---
class:primary
# Coming soon for 'You Draw It'\!

**Software Development**

<i class="fab fa-r-project" role="presentation" aria-label="r-project icon"></i> R package designed for easy implementation of ‘You Draw It’ task plots.

**Future Applications**

<i class="fa fa-chart-line" role="presentation" aria-label="chart-line icon"></i> Evaluate human's ability to make future predictions from trends and to fit curved trend lines.

<i class="fa fa-cloud" role="presentation" aria-label="cloud icon"></i> Apply 'You Draw It' to other fields in order to evaluate science communication using graphics.

**Publications**

<i class="fab fa-readme" role="presentation" aria-label="readme icon"></i> [Under Review] Robinson, E. A., VanderPlas, S., Howard, R., (2021) Eye Fitting Straight Lines in the Modern Era. Submitted to *Journal of Computational and Graphical Statistics*. [Access on GitHub.](https://earobinson95.github.io/Eye-Fitting-Straight-Lines-in-the-Modern-Era/Eye-Fitting-Straight-Lines-in-the-Modern-Era.pdf)

???

While some technical details were presented here, I intend to create an R package designed for easy implementation of ‘You Draw It’ task plots in order to make this tool accessible to other researchers. Further investigation is necessary to implement this method with real data in order to facilitate scientific communication and to evaluate the ability to make future predictions and looka t forcasting biases.

---
class:primary
# References
<font size="3">
<p><cite><a id='bib-aisch_cox_quealy_2015'></a><a href="#cite-aisch_cox_quealy_2015">Aisch, G., A. Cox, and K. Quealy</a>
(2015).
<em>You Draw It: How Family Income Predicts Children's College Chances</em>.
URL: <a href="https://www.nytimes.com/interactive/2015/05/28/upshot/you-draw-it-how-family-income-affects-childrens-college-chances.html">https://www.nytimes.com/interactive/2015/05/28/upshot/you-draw-it-how-family-income-affects-childrens-college-chances.html</a>.</cite></p>

<p><cite><a id='bib-buchanan_park_pearce_2017'></a><a href="#cite-buchanan_park_pearce_2017">Buchanan, L., H. Park, and A. Pearce</a>
(2017).
<em>You Draw It: What Got Better or Worse During Obama's Presidency</em>.
URL: <a href="https://www.nytimes.com/interactive/2017/01/15/us/politics/you-draw-obama-legacy.html">https://www.nytimes.com/interactive/2017/01/15/us/politics/you-draw-obama-legacy.html</a>.</cite></p>

<p><cite><a id='bib-ciccione2021'></a><a href="#cite-ciccione2021">Ciccione, L. and S. Dehaene</a>
(2021).
&ldquo;Can humans perform mental regression on a graph? Accuracy and bias in the perception of scatterplots&rdquo;.
In: <em>Cognitive Psychology</em> 128, p. 101406.</cite></p>

<p><cite><a id='bib-finney1951subjective'></a><a href="#cite-finney1951subjective">Finney, D.</a>
(1951).
&ldquo;Subjective judgment in statistical analysis: An experimental study&rdquo;.
In: <em>Journal of the Royal Statistical Society: Series B (Methodological)</em> 13.2, pp. 284&ndash;297.</cite></p>

<p><cite><a id='bib-katz_2017'></a><a href="#cite-katz_2017">Katz, J.</a>
(2017).
<em>You Draw It: Just How Bad Is the Drug Overdose Epidemic?</em>
URL: <a href="https://www.nytimes.com/interactive/2017/04/14/upshot/drug-overdose-epidemic-you-draw-it.html">https://www.nytimes.com/interactive/2017/04/14/upshot/drug-overdose-epidemic-you-draw-it.html</a>.</cite></p>

<p><cite><a id='bib-mosteller1981eye'></a><a href="#cite-mosteller1981eye">Mosteller, F., A. F. Siegel, E. Trapido, et al.</a>
(1981).
&ldquo;Eye fitting straight lines&rdquo;.
In: <em>The American Statistician</em> 35.3, pp. 150&ndash;152.</cite></p>

<p><cite><a id='bib-r2d3_pkg'></a><a href="#cite-r2d3_pkg">Strayer, N., J. Luraschi, and J. Allaire</a>
(2020).
<em>r2d3: Interface to 'D3' Visualizations</em>.
R package version 0.2.5.
URL: <a href="https://CRAN.R-project.org/package=r2d3">https://CRAN.R-project.org/package=r2d3</a>.</cite></p>
</font>

---
class:inverse
<br>
<br>
<br>
.center[
# Thank you! 
### Acknowledgments: Dr. Susan VanderPlas and Dr. Reka Howard
<br>
<br>
<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:white;overflow:visible;position:relative;"><path d="M502.3 190.8c3.9-3.1 9.7-.2 9.7 4.7V400c0 26.5-21.5 48-48 48H48c-26.5 0-48-21.5-48-48V195.6c0-5 5.7-7.8 9.7-4.7 22.4 17.4 52.1 39.5 154.1 113.6 21.1 15.4 56.7 47.8 92.2 47.6 35.7.3 72-32.8 92.3-47.6 102-74.1 131.6-96.3 154-113.7zM256 320c23.2.4 56.6-29.2 73.4-41.4 132.7-96.3 142.8-104.7 173.4-128.7 5.8-4.5 9.2-11.5 9.2-18.9v-19c0-26.5-21.5-48-48-48H48C21.5 64 0 85.5 0 112v19c0 7.4 3.4 14.3 9.2 18.9 30.6 23.9 40.7 32.4 173.4 128.7 16.8 12.2 50.2 41.8 73.4 41.4z"/></svg> **emily.robinson@huskers.unl.edu**
<svg aria-hidden="true" role="img" viewBox="0 0 576 512" style="height:1em;width:1.12em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:white;overflow:visible;position:relative;"><path d="M280.37 148.26L96 300.11V464a16 16 0 0 0 16 16l112.06-.29a16 16 0 0 0 15.92-16V368a16 16 0 0 1 16-16h64a16 16 0 0 1 16 16v95.64a16 16 0 0 0 16 16.05L464 480a16 16 0 0 0 16-16V300L295.67 148.26a12.19 12.19 0 0 0-15.3 0zM571.6 251.47L488 182.56V44.05a12 12 0 0 0-12-12h-56a12 12 0 0 0-12 12v72.61L318.47 43a48 48 0 0 0-61 0L4.34 251.47a12 12 0 0 0-1.6 16.9l25.5 31A12 12 0 0 0 45.15 301l235.22-193.74a12.19 12.19 0 0 1 15.3 0L530.9 301a12 12 0 0 0 16.9-1.6l25.5-31a12 12 0 0 0-1.7-16.93z"/></svg> **www.emilyarobinson.com**
<svg aria-hidden="true" role="img" viewBox="0 0 496 512" style="height:1em;width:0.97em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:white;overflow:visible;position:relative;"><path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"/></svg> **earobinson95**
]

???

Big thanks to my advisers at UNL, Dr. Susan VanderPlas and Dr. Reka Howard, for guiding me through this research.

---
class:primary
# Code Sketch (detailed)

<br>
.center[ 
 <img src="images/r2d3+shiny.png" width="100%"/>
 
]

---
class:primary
# Feedback Data Notation

.pull-left[
For each participant, the final data set used for analysis contains:
+ `\(x_{ijk}\)`, `\(y_{ijk,drawn}\)`, `\(\hat y_{ijk,OLS}\)`, `\(\hat y_{ijk,PCA}\)`

for 
+ parameter choice `\(i = 1,2,3,4\)`,
+ participant j = `\(1,...N_{participant}\)` 
+ `\(x_{ijk}\)` value corresponding to increment `\(k = 1, ...,4 x_{max} + 1\)`.
].pull-right[

<img id='imgeyefittingtrialplot2' src='images/eyefitting-trial-plot-2.png' alt=' ' width='80%'>

<div id='modaleyefittingtrialplot2' class='modal'>
  
  <img class='modal-content' id='imgmodaleyefittingtrialplot2'>
  
  <div id='captioneyefittingtrialplot2' class='modal-caption'></div>
</div>
]

**Vertical residuals** between the drawn and fitted values were calculated as:
+ `\(e_{ijk,OLS} = y_{ijk,drawn} - \hat y_{ijk,OLS}\)`
+ `\(e_{ijk,PCA} = y_{ijk,drawn} - \hat y_{ijk,PCA}\)`.

---
class:primary
# LMM Equation

The **Linear Mixed Model** equation for each fit (OLS and PCA) residuals is given by:
`\begin{equation}
e_{ijk,fit} = \left[\gamma_0 + \alpha_i\right] + \left[\gamma_{1} x_{ijk} + \gamma_{2i} x_{ijk}\right] + p_{j} + \epsilon_{ijk}
\end{equation}`
where

+ `\(e_{ijk,fit}\)` is the residual between the drawn and fitted y-values for the `\(i^{th}\)` parameter choice, `\(j^{th}\)` participant, and `\(k^{th}\)` increment of x-value corresponding to either the OLS or PCA fit
+ `\(\gamma_0\)` is the overall intercept
+ `\(\alpha_i\)` is the effect of the `\(i^{th}\)` parameter choice (F, S, V, N) on the intercept
+ `\(\gamma_1\)` is the overall slope for `\(x\)`
+ `\(\gamma_{2i}\)` is the effect of the parameter choice on the slope
+ `\(x_{ijk}\)` is the x-value for the `\(i^{th}\)` parameter choice, `\(j^{th}\)` participant, and `\(k^{th}\)` increment
+ `\(p_{j} \sim N(0, \sigma^2_{participant})\)` is the random error due to the `\(j^{th}\)` participant's characteristics
+ `\(\epsilon_{ijk} \sim N(0, \sigma^2)\)` is the residual error.

???

Using the `lmer` function in the lme4 package, a linear mixed model (LMM) is fit separately to the OLS and PCA residuals, constraining the fit to a linear trend.

---
class:primary
# GAMM Equation

The **Generalized Additive Mixed Model** equation for each fit (OLS and PCA) residuals is given by: 
`\begin{equation}
e_{ijk,fit} = \alpha_i + s_{i}(x_{ijk}) + p_{j} + s_{j}(x_{ijk})
\end{equation}`
where

+ `\(e_{ijk,fit}\)` is the residual between the drawn and fitted y-values for the `\(i^{th}\)` parameter choice, `\(j^{th}\)` participant, and `\(k^{th}\)` increment of x-value corresponding to either the OLS or PCA fit
+ `\(\alpha_i\)` is the intercept for the parameter choice `\(i\)`
+ `\(s_{i}\)` is the smoothing spline for the `\(i^{th}\)` parameter choice
+ `\(x_{ijk}\)` is the x-value for the `\(i^{th}\)` parameter choice, `\(j^{th}\)` participant, and `\(k^{th}\)` increment
+ `\(p_{j} \sim N(0, \sigma^2_{participant})\)` is the error due to participant variation
+ `\(s_{j}\)` is the random smoothing spline for each participant.

???

Eliminating the linear trend constraint, the `bam` function in the mgcv package is used to fit a generalized additive mixed model (GAMM) separately to the OLS and PCA residuals to allow for estimation of smoothing splines.

---
class:inverse
<br>
<br>
<br>
<br>
<br>
<br>
<br>
.center[
# Exponential Growth Study
]

---
class:primary 
# Exponential Growth 'You Draw It'

Study Participant Prompt: *Use your mouse to fill in the trend in the yellow box region.*

.center[
<img src="images/exponential_example.gif" width="55%" />
]
---
class:primary 
# Treatment design

.pull-left[
2 x 2 x 2 factorial:
+ **growth rate:** low and high.
+ **points truncated:** `\(50\%\)` and `\(75\%\)` of the domain.
+ **scale:** log and linear.

].pull-right[
.center[

<img id='imglow10linear2' src='images/low-10-linear-2.png' alt='Low Growth Rate, 50% Truncation, Linear Scale' width='30%'>

<img id='imglow10log2' src='images/low-10-log-2.png' alt='Low Growth Rate, 50% Truncation, Log Scale' width='30%'>

<div id='modallow10linear2' class='modal'>
  
  <img class='modal-content' id='imgmodallow10linear2'>
  
  <div id='captionlow10linear2' class='modal-caption'></div>
</div>

<div id='modallow10log2' class='modal'>
  
  <img class='modal-content' id='imgmodallow10log2'>
  
  <div id='captionlow10log2' class='modal-caption'></div>
</div>

<img id='imglow15linear2' src='images/low-15-linear-2.png' alt='Low Growth Rate, 75% Truncation, Linear Scale' width='30%'>

<img id='imglow15log2' src='images/low-15-log-2.png' alt='Low Growth Rate, 75% Truncation, Log Scale' width='30%'>

<div id='modallow15linear2' class='modal'>
  
  <img class='modal-content' id='imgmodallow15linear2'>
  
  <div id='captionlow15linear2' class='modal-caption'></div>
</div>

<div id='modallow15log2' class='modal'>
  
  <img class='modal-content' id='imgmodallow15log2'>
  
  <div id='captionlow15log2' class='modal-caption'></div>
</div>

<img id='imghigh10linear2' src='images/high-10-linear-2.png' alt='High Growth Rate, 50% Truncation, Linear Scale' width='30%'>

<img id='imghigh10log2' src='images/high-10-log-2.png' alt='High Growth Rate, 50% Truncation, Log Scale' width='30%'>

<div id='modalhigh10linear2' class='modal'>
  
  <img class='modal-content' id='imgmodalhigh10linear2'>
  
  <div id='captionhigh10linear2' class='modal-caption'></div>
</div>

<div id='modalhigh10log2' class='modal'>
  
  <img class='modal-content' id='imgmodalhigh10log2'>
  
  <div id='captionhigh10log2' class='modal-caption'></div>
</div>

<img id='imghigh15linear2' src='images/high-15-linear-2.png' alt='High Growth Rate, 75% Truncation, Linear Scale' width='30%'>

<img id='imghigh15log2' src='images/high-15-log-2.png' alt='High Growth Rate, 75% Truncation, Log Scale' width='30%'>

<div id='modalhigh15linear2' class='modal'>
  
  <img class='modal-content' id='imgmodalhigh15linear2'>
  
  <div id='captionhigh15linear2' class='modal-caption'></div>
</div>

<div id='modalhigh15log2' class='modal'>
  
  <img class='modal-content' id='imgmodalhigh15log2'>
  
  <div id='captionhigh15log2' class='modal-caption'></div>
</div>
]]

---
class:primary 
# Feedback data

.pull-left[
For each participant, the final data set used for analysis contains:
+ `\(x_{ijklm}\)`, `\(y_{ijklm,drawn}\)`, and `\(\hat y_{ijklm,NLS}\)`  
 
for:
+ growth rate `\(i = 1,2\)`, 
+ point truncation `\(j = 1,2\)`,
+ scale `\(k = 1,2\)`,
+ participant  `\(l = 1,...,N_\text{participant}\)`, and
+ `\(x_{ijklm}\)` value `\(m = 1, ...,4 x_\text{max} + 1\)`.

Vertical residuals between the drawn and fitted values were calculated as: 
+ `\(e_{ijklm,NLS} = y_{ijklm,drawn} - \hat y_{ijklm,NLS}\)`.
 
].pull-right[

<img id='imgexponentialyloessspaghettiplot21' src='images/exponential-yloess-spaghetti-plot-2-1.png' alt=' ' width='100%'>

<div id='modalexponentialyloessspaghettiplot21' class='modal'>
  
  <img class='modal-content' id='imgmodalexponentialyloessspaghettiplot21'>
  
  <div id='captionexponentialyloessspaghettiplot21' class='modal-caption'></div>
</div>
]

---
class:primary 
# Generalized Additive Mixed Model

The GAMM equation for residuals is given by:
`\begin{equation}
e_{ijklm,nls} = \tau_{ijk} + s_{ijk}(x_{ijklm}) + p_{l} + s_{l}(x_{ijklm})
\end{equation}`

where
 
+ `\(e_{ijklm,NLS}\)` is the residual between the drawn y-value and fitted y-value for the `\(l^{th}\)` participant, `\(m^{th}\)` increment, and `\(ijk^{th}\)` treatment combination 
+ `\(\tau_{ijk}\)` is the intercept for the `\(i^{th}\)` growth rate, `\(j^{th}\)` point truncation, and `\(k^{th}\)` scale treatment combination
+ `\(s_{ijk}\)` is the smoothing spline for the `\(ijk^{th}\)` treatment combination
+ `\(x_{ijklm}\)` is the x-value for the `\(l^{th}\)` participant, `\(m^{th}\)` increment, and `\(ijk^{th}\)` treatment combination 
+ `\(p_{l} \sim N(0, \sigma^2_{participant})\)` is the error due to the `\(l^{th}\)` participant's characteristics 
+ `\(s_{l}\)` is the random smoothing spline for the `\(l^{th}\)` participant.

???

Allowing for flexibility, the bam function in the mgcv package is used to fit a GAMM to estimate trends of vertical residuals from the participant drawn line in relation to the NLS fitted values.

---
class:primary
# GAMM results

.center[

<img id='imgexponentialpredictiongammpreds22' src='images/exponential-prediction-gamm-preds-2-2.png' alt=' ' width='64%'>

<div id='modalexponentialpredictiongammpreds22' class='modal'>
  
  <img class='modal-content' id='imgmodalexponentialpredictiongammpreds22'>
  
  <div id='captionexponentialpredictiongammpreds22' class='modal-caption'></div>
</div>

<img id='imgexponentialpredictiongammpreds23' src='images/exponential-prediction-gamm-preds-2-3.png' alt=' ' width='64%'>

<div id='modalexponentialpredictiongammpreds23' class='modal'>
  
  <img class='modal-content' id='imgmodalexponentialpredictiongammpreds23'>
  
  <div id='captionexponentialpredictiongammpreds23' class='modal-caption'></div>
</div>
]