Assignment due date: 28 Jan 2019

Introduction

A research team investigates selection performance with two types of menus:

  1. UNORDERED. The menu items in these menus are randomly ordered.

  2. ORDERED. The menu items in these menus are ordered alphabetically.

The researchers hypothesize that selection time is faster with alphabetically ordered menus. To test this hypothesis, they run an experiment with 18 participants. The researchers follow a within-participants (repeated-measures) design, where all 18 participants test both menu types. For each menu type, each participant performs 30 trials. Thus, each participant performs a total of \(2 \times 30 = 60\) trials, while the total number of experimental trials is \(18 \times 2 \times 30 = 1080\).

For each trial, the participant is shown the name of a menu item (e.g., “elephant”) and is required to first click to open the menu, find the correct menu item (“elephant”) within this menu, and click on it. The researchers measure the time (in msec) it takes to complete the task. It is measured from the time the participant clicks on the menu to open it to the time the participant clicks on the correct menu item.

To minimize order effects, the researchers randomly split participants into two equal groups, where each group is exposed to the two menu types in a different order.

Step 1. Data collection

Suppose the researchers have completed the experiment and have collected the results. Here, we simulate the data generation process by randomly sampling from some fixed populations. To create your data file, you will use this RScript. You must simply run the script and then use the produced dataset.cvs file as your dataset. Notice that each of you will generate a different dataset, so the results of your analyses might be different.

The file should contain four columns with the following information: (1) the participant number, (2) the type of menu, (3) the number of trial, and (4) the task-completion time (in msec).

Step 2. Descriptive statistics

Write an R script to calculate the descriptive statistics for your experimental data. The descriptive statistics will include means, medians, and standard deviations for each individual type of menu (UNORDERED and ORDERED).

Step 3. Plots

Produce a box plot to graphically summarize your data for both menu types. Also produce a qqplot to visually assess the extent to which your data deviate from normality. What do you observe?

Step 4. Normal distributions

Assume populations with normal distributions. Write an R script that estimates the mean selection time for each type of menu, as well as for their mean time difference. Use 95% confidence intervals to estimate those means. The R script should include code that calculates and also graphically plots the confidence intervals. What are your conclusions?

Step 5. Log-Normal distributions

The researchers suspect that distributions are not normal and decide to conduct their analysis based on log-normal distributions. Describe the steps of this analysis. Write an R script that constructs 95% confidence intervals to estimate the median selection time for each menu type but also to compare their performance (Note: The comparison will now be based on a ratio rather than a difference and medians rather than means). Do your conclusions change?

Step 6. Bootstrapping

Use the percentile bootstrap method to construct the confidence intervals described at Step 4 (mean selection time for each type of menu, as well as for their mean time difference). What do you observe? Do your conclusions change?

Step 7. Report

Write an R Markdown page to describe your solutions. This page will include (i) your R scripts, (ii) their outputs, (iii) sufficient text to explain your steps, and (iv) your conclusions.

WHAT TO SUBMIT: Your data file (dataset.csv) and the HTML page generated from your R Markdown code.

ADVICE: You are encouraged to discuss the problems and their solutions with your colleagues and with your instructor (e.g., on Slack). However, your final solutions and report is personal.