Stat 263/363: Experimental Design

Overview

Correlation is not causality. You've probably heard that before in any number of regression classes. If you want to infer causality from data, then the best way is to use randomized experiments. Maybe it is the only way to be sure.

In experimental design we look at how to choose the data that we will gather. In addition to being able to make causal conclusions, we also look at how to maximize the statistical efficiency of the generated data set.

Experimental design as a subject is about 100 years old. The methods in this course date back to agricultural field trials. Since then the ideas have seen use in medicine, manufacturing, quality control, computer aided design and electronic commerce. Each new field takes the previous methods and then starts adapting them. Possibly the first clinical trial was that of James Lind in 1747 showing that citrus is effective against scurvy. (It was not immediately adopted.)

There will be about 3 problem sets and a project. The project will involve designing, carrying out and analyzing a real experiment. This can be from your every day life: cooking, hobbies, exercise routines, etc.

This Mark Rober video describes an experiment to study which animals (snake vs turtle vs tarantula) are more likely to be run over by vehicles. The results are interesting. It is also funny. (I don't recognize the characters that appear near the end though.)

Goals

1. Learn the main/classical methods of experimental design so that when it comes time to gather data you can work out the right choice.
2. Exposure to the research frontier in DoE: kriging, online experiments, design for high dimensional regression.
3. Do a designed statistical experiment from conception to execution to analysis.

Topics

• Experiments vs observation. Confounding
• Randomization and ANOVA
• Online A/B tests and bandits
• Neyman-Rubin causal model
• Blocking (basic)
• Blocking (advanced) Latin squares and related methods
• Factorial and fractional factorial experiments
• Split plots
• Response surfaces, central composite designs, Box-Behnken
• Optimal design
• Taguchi methods
• Mixture designs
• Computer experiments, space filling designs and kriging

Classes

Gates B12 Mon & Wed 3:00 to 4:20

Instructor

Art Owen

Sequoia Hall 130

My userid is owenpelican on stanfordpelican.edu (remember to remove the waterfowl)

Office hour: Friday 11:00-12:00

TAs

• Minyong Lee minyongPenguin@stanford.edu Monday 10-12, Sequoia Hall Fishbowl
• Joshua Loftus joftiusPenguin@stanford.edu Wednesday, Thursday 10-11, Sequoia Hall 240

Delete any and all Antarctic birds from the TA's email

Texts

Our main text is "Design and analysis of experiments" by D. C. Montgomery. It covers the `goal 1' material above. For goal 2 we will use supplementary readings.

Supplementary readings

• Paul Holland's article on Neyman-Rubin causality.
• Kohavi, Henne and Sommerfield's article on A/B tests for e-commerce.
• Agrawal and Goyal's article on Thompson sampling
• Bubeck and Cesa-Bianchi's monograph on bandit problems.
• Wikipedia entry on Hadamard matrices for Plackett-Burmann designs
• S. Georgiou's review of supersaturated designs.
• Krahmen and Ward's review of design for compressed sensing
• Dean Eckles' thesis including a network experiment using Facebook data

I expect to send a small number of important emails about problem sets and the homework there. Most other announcements will be made in class. If you email me about the class, be sure to have stat 363 or stat 263 in your subject line. Otherwise, your email won't show when I search for course related emails.

We will count days late on each problem set. Each day late is penalized by 10% of the homework value. Homework more than 3 days late will ordinarily get 0. If you're travelling, you can email a pdf file. For sickness, interviews and other events, up to 3 late days total are forgiven at the end of the quarter. (Work late enough to get zero does not get redeemed though.)