Loading the data

Recall from last week that you can download the checkpoints_eoc data with the following:

checkpoints_eoc <-read.csv("https://www.dropbox.com/scl/fi/3xda6hny48ojknoifupul/checkpoints_eoc.csv?rlkey=q6rksum9wc91pkte2qdg31s33&st=fr0rynut&dl=1")

Today, we are going to add a second data set from the same provider. You can download this second data set, checkpoints_pulse, using:

checkpoints_pulse <- read.csv("https://sta175.github.io/class_activities/data/checkpoints_pulse.csv")

The first part of today’s activity will focus on the new checkpoints_pulse data. In the second part, you will combine the two datasets.

Data citation

The data we are working with are provided by CourseKata, creators of an online interactive statistics textbook.

CourseKata provided the data for DataFest 2024, and we are now working with a publicly available version of these data downloaded from https://github.com/coursekata/data

• Citation for the Book: Son, J.Y. & Stigler, J.W. (2017-2024). Statistics and data science: A modeling approach. Los Angeles: CourseKata, https://coursekata.org/preview/default/program . Currently available in 7 versions.

New Data Description

The checkpoints_pulse dataset consists of 76848 rows and 8 columns, and the data records student responses to four “pulse” questions which are asked at the start of each chapter (except chapter 1), and which are designed to measure how students feel about the course, material, and textbook. Each question is scored on a scale of 1 (strongly disagree) to 6 (strongly agree). These questions are:

• Cost: I was unable to put in the time needed to do well in the previous chapter
• Expectancy: I am confident about what I learned in the previous chapter
• Intrinsic Value: I think this class is interesting
• Utility Value: I think what I have learned in the previous chapter is useful

The columns are:

• book - The title of the specific book used by the student. CourseKata releases multiple books, generally each targeting a specific age or skill level
• release - The release of the specific book used by the student. Think of this like a version number, as some versions of the textbook are newer than others.
• instituion_id - The institution at which the student was enrolled; this is as specific as Wake Forest, but recorded as a number
• class_id - The class the student was enrolled in; this is as specific as Dr. Dalzell’s STA 279 class in Spring 2025, but recorded as a number.
• student_id - The unique ID number of the student.
• chapter_num - The chapter of the book
• construct - The question the student is responding to (Cost, Expectancy, Intrinsic Value, Utility Value)
• response - The student’s response (an integer between 1 and 6, or NA if missing)

Dividing work between team members

When you have several different research questions you are trying to answer, you may wish to divide them between your different team members. Each team member can tackle a different question, and share their results with the group. If you find anything interesting or unusual about the data, make sure to tell your teammate – it may be important for them to know too!

Questions

Okay, now that we’ve talked about the tips, it’s time to answer the client’s research questions. Remember, there is more than one correct way to answer, so explore a bit.

Reshaping data

The fourth question from the client asks us to explore the relationship between different constructs. This is usually easiest when we have a column for each construct (cost, expectancy, etc) in the data. Right now, however, we have a single column for construct, with different rows for the different constructs.

In order to answer the fourth research question, it may be helpful to change the shape of the data, by converting the single construct column into separate columns for each construct. In R, this is called pivoting, and the tidyr package provides two pivoting functions: pivot_wider and pivot_longer. We want to make “long” data “wider” by adding columns, so we will use the pivot_wider function.

You will see more data wrangling in a later activity. For now, here is the code that allows us to pivot the data in this way:

pulse_pivot <- checkpoints_pulse |>
  pivot_wider(id_cols = c(book, release, institution_id,
                          class_id, student_id, chapter_num),
              names_from = construct,
              values_from = response,
              values_fn = function(x){mean(x, na.rm=T)})

Questions

Joining datasets

Combining multiple datasets is a common task in DataFest, in which you will often receive several different files, recording different but related pieces of information.

If we examine the checkpoints_eoc data, we see that there is one row for each combination of student and chapter. The checkpoints_pulse data, however, has one row for each combination of student, chapter, and construct. Instead, we will use the pulse_pivot dataframe we created above, and combine it with the checkpoints_eoc dataframe.

The two dataframes have different numbers of rows, and the rows are probably not in the same order. So how should we decide how to join them? The crucial point is that each row is identified uniquely by a set of variables: book, release, class, student, and chapter. We will match rows from pulse_pivot to rows from checkpoints_eoc using these variables.

There is also one more wrinkle: questions at the beginning of a chapter really refer to student’s feelings about the previous chapter. To match chapter experiences, we will therefore change the chapter numberings in the “pulse” questions; e.g., “pulse” questions answered in chapter 2 really refer to chapter 1.

Here is the code in R:

checkpoints_combined <- pulse_pivot |>
  mutate(chapter_num = as.numeric(chapter_num) - 1) |>
  inner_join(checkpoints_eoc, join_by(book, release, class_id, student_id, 
                                      chapter_num))

Now if you look at the new checkpoints_combined dataframe, you will see that there are the columns from the pulse_pivot dataframe, AND the columns from checkpoints_eoc.

Questions

The fifth research question (“How are the”pulse” questions related to student performance in the end-of-chapter questions?“) is quite open-ended. Before we try to answer it, we should create more specific questions to answer. For example, we could hypothesize that when students perform poorly on end-of-chapter questions, they will also report lower interest in the course material.