Griffin R Workshop Notes
Learning Data Structures and Exploring Data Frames in R adapted from R Software Carpentry
Exploring R Data Structures
Setting up Directory and Creating a Toy Dataset
One of R’s most powerful features is its ability to deal with tabular data - such as you may already have in a spreadsheet or a CSV file.
Let’s start by creating a toy dataset of feline-data.csv in data/ directory (folder)
But if data/ directory does not exist, create it first.
Start with orienting yourself in the R environment. Where am I?
getwd() # prints the current working directory "where am I?"
dir.create("data") # creates a new directory called "data"
setwd("working/directory/data") # sets the working directory to the working directoryCheck if all sticky notes are green
Now, we are inside the data/ directory. We can now create a toy dataset of feline data and save it as feline-data.csv.
First, let’s create a data frame in R:
A data frame is a table or a two-dimensional array-like structure in which each column contains values of one variable and each row contains one set of values from each column. It is similar to a spreadsheet or SQL table, or a data frame in Python’s pandas library.
cats <- data.frame(coat = c("calico", "black", "tabby"),
weight = c(2.1, 5.0, 3.2),
likes_catnip = c(1, 0, 1))We can now save cats as a CSV file using the write.csv() function:
write.csv(cats, "feline-data.csv", row.names = FALSE) # it is important to call the argument row.names = FALSE to avoid writing row numbers to the CSV fileThe contents of the feline-data.csv file will look like this:
coat,weight,likes_catnip
calico,2.1,1
black,5.0,0
tabby,3.2,1We can now load this in our R environment using the read.csv() function:
cats <- read.csv(file = "data/feline-data.csv")
catsread.csv() is a function that reads a CSV file or comma-separated values file and creates a data frame in R. The file argument specifies the path to the CSV file. In this case, we are reading the feline-data.csv file from the data/ directory.
You can check out other function like read.delim for tab-separated values file but for now, we will stick with read.csv.
Check if all sticky notes are green
We should only see numeric and character data types. If you see factors, we can prevent R from automatically creating “factor” data by using the options(stringsAsFactors = FALSE) function. Then re-read the feline-data.csv file for the changes to take effect.
str(cats)Now, We can begin exploring our dataset, pulling out columns by specifying them using the $ operator:
cats$weight # prints the weight column
cats$coat # prints the coat columnWe can also do operation on the columns:
Say we discovered that the scale weighs two Kg lighter, then we have to add 2 Kg to the values in weight column.
cats$weight + 2 # does not change the original data frame, but prints the new values## do not run the code; show only!
cats$weight <- cats$weight + 2 # changes the original data frame by adding 2 Kg to the values in weight columnIf we wanted to write a sentence about our cats, we can use the paste() function to combine the values in the coat column with a string:
paste("My cat is", cats$coat)But what about adding the weight column to the coat column? Let’s try it:
cats$weight + cats$coatError in `cats$weight + cats$coat`:
! non-numeric argument to binary operatorUnderstanding what happened here is key to successfully analyzing data in R.
The error message is telling us that we cannot add a numeric vector (the weight column) to a character vector (the coat column). This is because R does not know how to combine these two different types of data.
Data Types
If you guessed that the last command will return an error because 2.1 plus black is nonsense, you’re right - and you already have some intuition about data types in R. Let’s explore this a bit more by checking the data type of the weight column:
typeof(cats$weight) There are five main types: double, integer, complex, logical and character. For historic reasons, double is also called numeric.
typeof(1L) # The L suffix forces the number to be an integer, since by default R uses float numberstypeof(1+1i)typeof(TRUE)typeof('banana')So no matter how complicated our analyses are, we will always be working with these five basic data types. The key is to understand how to manipulate them and combine them in ways that make sense for our analyses.
What if your labmate provided another details of another cat? We can add an additional row to our cats table using the rbind() function:
We assign the new row to a variable called additional_cat:
additional_cat <- data.frame(coat = "tabby", weight = "2.3 or 2.4", likes_catnip = 1)
additional_catThen we create a new data frame called cats2 by combining the original cats data frame with the additional_cat data frame using the rbind() function:
cats2 <- rbind(cats, additional_cat)
cats2Let’s check what type of data is in the weight column of the new cats2 data frame:
typeof(cats2$weight)It says character! Do you think we can still do the same math we did on them before? Let’s try it:
cats2$weight + 2Error in `cats2$weight + 2`:
! non-numeric argument to binary operatorAside from we cannot add a character type and numerical together, cats and cats2 are data frames. Data frames are one of the most common and versatile types of data structures we will work with in R. A given column in a data frame cannot be composed of different data types. In this case, R cannot store everything in the data frame column weight as a double anymore once we add the row for the additional cat (because its weight is 2.3 or 2.4).
When R reads a csv file, it reads it in as a data frame. Thus, when we loaded the cats csv file, it is stored as a data frame. We can recognize data frames by the first row that is written by the str() function:
str(cats2)'data.frame': 4 obs. of 3 variables:
$ coat : chr "calico" "black" "tabby" "tabby"
$ weight : chr "2.1" "5" "3.2" "2.3 or 2.4"
$ likes_catnip: num 1 0 1 1Data frames are composed of rows and columns, where each column has the same number of rows. Different columns in a data frame can be made up of different data types (this is what makes them so versatile), but everything in a given column needs to be the same type (e.g., vector, factor, or list).
Let’s explore more about different data structures and how they behave. For now, we will focus on our original data frame cats (and we can forget about cats2 for the rest of this lesson).
Vectors and Type Coercion
To better understand this behavior, let’s meet another of the data structures: the vector.
my_vector <- vector(length = 3)
my_vectorA vector in R is simply a collection of values of the same basic type stored together.
If no data type is specified, R creates a vector of type logical (TRUE/FALSE) when we use the vector() function.
Let’s try to create a vector of type character:
another_vector <- vector(mode='character', length=3)
another_vectorIt looks weird and doesn’t tell us much. Let’s check the structure using str function.
str(another_vector)Although it looks somewhat cryptic, this command indicates the basic data type found in this vector - in this case chr, character; an indication of the number of things in the vector - actually, the indexes of the vector, in this case [1:3]; and a few examples of what’s actually in the vector - in this case empty character strings. If we similarly do
str(cats$weight)we see that cats$weight is a vector, too - the columns of data we load into R data.frames are all vectors, and that’s the root of why R forces everything in a column to be the same basic data type.
By keeping everything in a column the same, we allow ourselves to make simple assumptions about our data; if you can interpret one entry in the column as a number, then you can interpret all of them as numbers, so we don’t have to check every time. This consistency is what people mean when they talk about clean data; in the long run, strict consistency goes a long way to making our lives easier in R.
Coercion by combining vectors
You can also make vectors with explicit contents with the combine function:
combine_vector <- c(2,6,3)
combine_vectorGiven what we’ve learned so far, what do you think the following will produce?
quiz_vector <- c(2,6,'3')This is something called type coercion, and it is the source of many surprises and the reason why we need to be aware of the basic data types and how R will interpret them. When R encounters a mix of types (here double and character) to be combined into a single vector, it will force them all to be the same type. Consider:
coercion_vector <- c('a', TRUE)
coercion_vectorThe type of hierarchy
The coercion rules go: logical -> integer -> double (“numeric”) -> complex -> character, where “->” can be read as are transformed into. For example, combining logical and character transforms the result to character:
c('a', TRUE) # Transforms it to characterA quick way to recognize character vectors is by the quotes that enclose them when they are printed.
You can try to force coercion against this flow using the as. functions:
character_vector_example <- c('0','2','4')
character_vector_examplecharacter_coerced_to_double <- as.double(character_vector_example)
character_coerced_to_doubledouble_coerced_to_logical <- as.logical(character_coerced_to_double)
double_coerced_to_logicalAs you can see some surprising things can happen if R forces one basic data type into another! So we have to understand how coercion works and make sure everything is the same type in your vectors and your columns of data.frames, or we will get a uncalled surprises!
But coercion can also be very useful! For example, in our cats data likes_catnip is numeric, but we know that the 1s and 0s actually represent TRUE and FALSE (a common way of representing them). We should use the logical datatype here, which has two states: TRUE or FALSE, which is exactly what our data represents. We can ‘coerce’ this column to be logical by using the as.logical function:
cats$likes_catnipcats$likes_catnip <- as.logical(cats$likes_catnip)
cats$likes_catnipCheck if all sticky notes are green
Using the object cats2:
# 1. Print the data
cats2
print(cats2)
# 2. Show an overview of the table with all data types
str(cats2)
# 3. The "weight" column has the incorrect data type "character".
# The correct data type is: "double/numerical".
# 4. Correct the 4th weight data point with the mean of the two given values
cats2$weight[4] <- 2.35
# print the data again to see the effect
cats2
# 5. Convert the weight to the right data type
cats2$weight <- as.double(cats2$weight)
# Calculate the mean to test yourself
mean(cats2$weight)
## [1] 3.1625
# If you see the correct mean value (and not NA), you did the exercise correctly!List
Another data structure you’ll want to be aware of is the list. A list is simpler in some ways than the other types, because you can put anything you want in it. Remember everything in the vector must be of the same basic data type, but a list can have different data types:
list_example <- list(1, "a", TRUE, 1+4i)
list_exampleWhen printing the object structure with str(), we see the data types of all elements:
str(list_example)What is the use of lists? They can organize data of different types. For example, you can organize different tables that belong together, similar to spreadsheets in Excel. But there are many other uses, too.
We will see another example that will maybe surprise you in the next chapter.
To retrieve one of the elements of a list, use the double bracket:
list_example[[2]]The elements of lists also can have names, they can be given by prepending them to the values, separated by an equals sign:
another_list <- list(title = "Numbers", numbers = 1:10, data = TRUE )
another_listThis results in a named list. Now we have a new function of our object! We can access single elements by an additional way!
another_list$titleNames
With names, we can give meaning to elements. It is the first time that we do not only have the data, but also explaining information. It is metadata that can be stuck to the object like a label. In R, this is called an attribute. Some attributes enable us to do more with our object, for example, like here, accessing an element by a self-defined name.
Accessing vectors and lists by name
We have already seen how to generate a named list. The way to generate a named vector is very similar. You have seen this function before:
pizza_price <- c( pizzasubito = 5.64, pizzafresh = 6.60, callapizza = 4.50 )The way to retrieve elements is different, though:
pizza_price["pizzasubito"]The approach used for the list does not work:
pizza_price$pizzafreshIt means that you have just tried accessing an element like it was in a list, but it is actually in a vector.
Accessing and changing names
If you are only interested in the names, use the names() function:
names(pizza_price)We have seen how to access and change single elements of a vector. The same is possible for names:
names(pizza_price)[3]names(pizza_price)[3] <- "call-a-pizza"
pizza_priceWhat is the data type of the names of pizza_price? You can find out using the str() or typeof() functions.
typeof(names(pizza)) alternatively, use a new variable if this is easier for you to read:
n <- names(pizza)
typeof(n)Data Frames
We have data frames at the very beginning of this lesson, they represent a table of data. We didn’t go much further into detail with our example cat data frame:
catsWe can now understand something a bit surprising in our data.frame; what happens if we run:
typeof(cats)We see that data.frames look like lists ‘under the hood’. Think again what we heard about what lists can be used for:
Lists organize data of different types
Columns of a data frame are vectors of different types, that are organized by belonging to the same table.
A data.frame is really a list of vectors. It is a special list in which all the vectors must have the same length.
How is this “special”-ness written into the object, so that R does not treat it like any other list, but as a table?
class(cats)A class, just like names, is an attribute attached to the object. It tells us what this object means for humans.
You might wonder: Why do we need another what-type-of-object-is-this-function? We already have typeof()? That function tells us how the object is constructed in the computer. The class is the meaning of the object for humans. Consequently, what typeof() returns is fixed in R (mainly the five data types), whereas the output of class() is diverse and extendable by R packages.
In our cats example, we have an integer, a double and a logical variable. As we have seen already, each column of data.frame is a vector.
cats$coatcats[,1]typeof(cats[,1])str(cats[,1])Each row is an observation of different variables, itself a data.frame, and thus can be composed of elements of different types.
cats[1,]typeof(cats[1,])str(cats[1,])There are several subtly different ways to call variables, observations, and elements from data.frames:
cats[1]
cats[[1]]
cats$coat
cats["coat"]
cats[1, 1]
cats[, 1]
cats[1, ]Try out these examples and explain what is returned by each one.
Hint: Use the function typeof() to examine what is returned in each case.
typeof(names(pizza)) alternatively, use a new variable if this is easier for you to read:
n <- names(pizza)
typeof(n)Matrices
Last but not least is the matrix. We can declare a matrix full of zeros:
matrix_example <- matrix(0, ncol=6, nrow=3)
matrix_exampleWhat makes it special is the dim() attribute:
dim(matrix_example)And similar to other data structures, we can ask things about our matrix:
typeof(matrix_example)str(matrix_example)nrow(matrix_example)ncol(matrix_example)::: ## Challenge 4 4.1. What do you think will be the result of length(matrix_example)? Try it. Were you right? Why / why not?
Because a matrix is a vector with added dimension attributes, length gives you the total number of elements in the matrix.
4.2. Make another matrix, this time containing the numbers 1:50, with 5 columns and 10 rows. Did the matrix function fill your matrix by column, or by row, as its default behaviour? See if you can figure out how to change this. (hint: read the documentation for matrix!)
x <- matrix(1:50, ncol=5, nrow=10)
x <- matrix(1:50, ncol=5, nrow=10, byrow = TRUE) # to fill by row4.3. Consider the R output of the matrix below:
[,1] [,2]
[1,] 4 1
[2,] 9 5
[3,] 10 7What was the correct command used to write this matrix? Examine each command and try to figure out the correct one before typing them. Think about what matrices the other commands will produce.
matrix(c(4, 1, 9, 5, 10, 7), nrow = 3)
matrix(c(4, 9, 10, 1, 5, 7), ncol = 2, byrow = TRUE)
matrix(c(4, 9, 10, 1, 5, 7), nrow = 2)
matrix(c(4, 1, 9, 5, 10, 7), ncol = 2, byrow = TRUE)matrix(c(4, 1, 9, 5, 10, 7), ncol = 2, byrow = TRUE) # answer:::