Are Lectures Effective?

Setup

We start by loading R libraries we will use:

library(tidyverse)

Loading tidyverse: ggplot2
Loading tidyverse: tibble
Loading tidyverse: tidyr
Loading tidyverse: readr
Loading tidyverse: purrr
Loading tidyverse: dplyr
Conflicts with tidy packages ---------------------------------------------------
filter(): dplyr, stats
lag():    dplyr, stats

And set some R options:

options(repr.matrix.max.rows=20, repr.matrix.max.cols=20)
options(repr.plot.height=5)

Load Data

exam_perf = read_csv("tbl-exam-perf.csv")
head(exam_perf)

Parsed with column specification:
cols(
  Study = col_character(),
  Discipline = col_character(),
  Assessment = col_character(),
  Size = col_character(),
  Studnt = col_character(),
  Instr = col_character(),
  Maj = col_character(),
  Level = col_character(),
  Treatment = col_character(),
  `Sign?` = col_character(),
  OrigStN = col_character(),
  ClustAdjN = col_character(),
  HedgeG = col_double(),
  SE = col_double(),
  Lower = col_double(),
  Upper = col_double(),
  Z = col_double(),
  P = col_double()
)

Study	Discipline	Assessment	Size	Studnt	Instr	Maj	Level	Treatment	Sign?	OrigStN	ClustAdjN	HedgeG	SE	Lower	Upper	Z	P
(S53)Anderson et al. 20052	Chem	Exam	Sm	QRno	yes	Maj	Upper	PBL	Y	420	9	1.334	0.609	0.140	2.529	2.189	0.029
(S54)Armbrusteret al. 20091	Bio	Exam	Lg	QRno	yes	Maj	Intro	Wrksht, Clicker	Y	520	13	0.186	0.540	-0.872	1.244	0.345	0.730
(S55)Armstrong & Hendricks 1999	Math	(grade)	na	QRacad	yes	Maj	Intro	Wrkshts	N	14	na	0.322	0.655	-0.961	1.605	0.491	0.623
(S56)Armstrong et al. 2007	Bio	Exam	Lg	QRpre	yes	Non	Intro	Clicker, Wrksht, Quiz	N	614	9	-0.090	0.593	-1.252	1.073	-0.151	0.880
(S57)Asiala et al. 1997a2	Math	Exam	na	QRacad	no	Maj	Intro	Wrksht	na	41	8	0.817	0.633	-0.423	2.058	1.291	0.197
(S58)Asiala et al. 1997b	Math	Exam	na	QRacad	no	Maj	na	Wrksht	na	37	8	1.143	0.645	-0.121	2.406	1.772	0.076

Explore the Data

What are the different assessment types?

exam_perf %>%
    group_by(Assessment) %>%
    summarize(NStudies = n(), AvgImprovement=mean(HedgeG))

Assessment	NStudies	AvgImprovement
(grade)	14	0.2309286
(points)	9	0.2388889
CI	24	0.9655417
Exam	111	0.4288919

Let's look at the distribution of improvements:

ggplot(exam_perf) +
    aes(x=HedgeG) +
    geom_vline(xintercept=0, color="blue") +
    geom_histogram() +
    facet_wrap(~ Assessment, scales="free")

`stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

err = function(x) {
    n = length(x)
    qt(0.975,df=n-1)*sd(x)/sqrt(n)
}

assess_means = exam_perf %>%
    group_by(Discipline) %>%
    summarize(AvgImp = mean(HedgeG), AvgLow = mean(Lower), AvgHigh=mean(Upper),
              LB = mean(HedgeG) - err(HedgeG),
              UB = mean(HedgeG) + err(HedgeG))
assess_means

Discipline	AvgImp	AvgLow	AvgHigh	LB	UB
Bio	0.3001212	-0.7973939	1.397606	0.1592475	0.4409949
Chem	0.4040455	-0.7017273	1.509909	0.1789147	0.6291762
CompSci	0.3468750	-0.8396250	1.533500	-0.2686795	0.9624295
Eng	0.5646316	0.5264737	1.641053	0.1664520	0.9628112
Geo	0.5190000	-0.6555000	1.694000	-3.0514435	4.0894435
Math	0.3247586	-0.7842069	1.433759	0.1790062	0.4705110
Phys	0.7949032	-0.2464839	1.836258	0.4381906	1.1516158
Psych	0.6263571	-0.2999286	1.552286	0.4089961	0.8437182

ggplot(assess_means) +
    aes(y=AvgImp, x=Discipline, ymin=LB, ymax=UB) +
    geom_hline(yintercept=0, color="red") +
    geom_point() + geom_errorbar(width=0.5)