19.6 ... (dot-dot-dot)

!!! is useful because it’s not uncommon to have a list of expressions that you want to insert into a call. It turns out that this pattern is common elsewhere. Take the following two motivating problems:

  • What do you do if the elements you want to put in ... are already stored in a list? For example, imagine you have a list of data frames that you want to rbind() together:

    dfs <- list(
  a = data.frame(x = 1, y = 2),
  b = data.frame(x = 3, y = 4)
)

    You could solve this specific case with rbind(dfs$a, dfs$b), but how do you generalise that solution to a list of arbitrary length?

  • What do you do if you want to supply the argument name indirectly? For example, imagine you want to create a single column data frame where the name of the column is specified in a variable:

    var <- "x"
val <- c(4, 3, 9)

    In this case, you could create a data frame and then change names (i.e. setNames(data.frame(val), var)), but this feels inelegant. How can we do better?

One way to think about these problems is to draw explicit parallels to quasiquotation:

  • Row-binding multiple data frames is like unquote-splicing: we want to inline individual elements of the list into the call:

    dplyr::bind_rows(!!!dfs)
#>   x y
#> 1 1 2
#> 2 3 4

    When used in this context, the behaviour of !!! is known as “spatting” in Ruby, Go, PHP, and Julia. It is closely related to *args (star-args) and **kwarg (star-star-kwargs) in Python, which are sometimes called argument unpacking.

  • The second problem is like unquoting the left-hand side of =: rather than interpreting var literally, we want to use the value stored in the variable called var:

    tibble::tibble(!!var := val)
#> # A tibble: 3 x 1
#>       x
#>   <dbl>
#> 1     4
#> 2     3
#> 3     9

    Note the use of := (pronounced colon-equals) rather than =. Unfortunately we need this new operation because R’s grammar does not allow expressions as argument names:

    tibble::tibble(!!var = value)
#> Error: unexpected '=' in "tibble::tibble(!!var ="

    := is like a vestigial organ: it’s recognised by R’s parser, but it doesn’t have any code associated with it. It looks like an = but allows expressions on either side, making it a more flexible alternative to =. It is used in data.table for similar reasons.

Base R takes a different approach, which we’ll come back to in Section 19.6.4.

We say functions that support these tools, without quoting arguments, have tidy dots71. To gain tidy dots behaviour in your own function, all you need to do is use list2().

19.6.1 Examples

One place we could use list2() is to create a wrapper around attributes() that allows us to set attributes flexibly:

set_attr <- function(.x, ...) {
  attr <- rlang::list2(...)
  attributes(.x) <- attr
  .x
}

attrs <- list(x = 1, y = 2)
attr_name <- "z"

1:10 %>%
  set_attr(w = 0, !!!attrs, !!attr_name := 3) %>% 
  str()
#>  int [1:10] 1 2 3 4 5 6 7 8 9 10
#>  - attr(*, "w")= num 0
#>  - attr(*, "x")= num 1
#>  - attr(*, "y")= num 2
#>  - attr(*, "z")= num 3

19.6.2 exec()

What if you want to use this technique with a function that doesn’t have tidy dots? One option is to use rlang::exec() to call a function with some arguments supplied directly (in ...) and others indirectly (in a list):

# Directly
exec("mean", x = 1:10, na.rm = TRUE, trim = 0.1)
#> [1] 5.5

# Indirectly
args <- list(x = 1:10, na.rm = TRUE, trim = 0.1)
exec("mean", !!!args)
#> [1] 5.5

# Mixed
params <- list(na.rm = TRUE, trim = 0.1)
exec("mean", x = 1:10, !!!params)
#> [1] 5.5

rlang::exec() also makes it possible to supply argument names indirectly:

arg_name <- "na.rm"
arg_val <- TRUE
exec("mean", 1:10, !!arg_name := arg_val)
#> [1] 5.5

And finally, it’s useful if you have a vector of function names or a list of functions that you want to call with the same arguments:

x <- c(runif(10), NA)
funs <- c("mean", "median", "sd")

purrr::map_dbl(funs, exec, x, na.rm = TRUE)
#> [1] 0.444 0.482 0.298

exec() is closely related to call2(); where call2() returns an expression, exec() evaluates it.

19.6.3 dots_list()

list2() provides one other handy feature: by default it will ignore any empty arguments at the end. This is useful in functions like tibble::tibble() because it means that you can easily change the order of variables without worrying about the final comma:

# Can easily move x to first entry:
tibble::tibble(
  y = 1:5,
  z = 3:-1,
  x = 5:1,
)

# Need to remove comma from z and add comma to x
data.frame(
  y = 1:5,
  z = 3:-1,
  x = 5:1
)

list2() is a wrapper around rlang::dots_list() with defaults set to the most commonly used settings. You can get more control by calling dots_list() directly:

  • .ignore_empty allows you to control exactly which arguments are ignored. The default ignores a single trailing argument to get the behaviour described above, but you can choose to ignore all missing arguments, or no missing arguments.

  • .homonyms controls what happens if multiple arguments use the same name:

    str(dots_list(x = 1, x = 2))
#> List of 2
#>  $ x: num 1
#>  $ x: num 2
str(dots_list(x = 1, x = 2, .homonyms = "first"))
#> List of 1
#>  $ x: num 1
str(dots_list(x = 1, x = 2, .homonyms = "last"))
#> List of 1
#>  $ x: num 2
str(dots_list(x = 1, x = 2, .homonyms = "error"))
#> Error: Arguments can't have the same name.
#> We found multiple arguments named `x` at positions 1 and 2

  • If there are empty arguments that are not ignored, .preserve_empty controls what to do with them. The default throws an error; setting .preserve_empty = TRUE instead returns missing symbols. This is useful if you’re using dots_list() to generate function calls.

19.6.4 With base R

Base R provides a Swiss army knife to solve these problems: do.call(). do.call() has two main arguments. The first argument, what, gives a function to call. The second argument, args, is a list of arguments to pass to that function, and so do.call("f", list(x, y, z)) is equivalent to f(x, y, z).

  • do.call() gives a straightforward solution to rbind()ing together many data frames:

    do.call("rbind", dfs)
#>   x y
#> a 1 2
#> b 3 4

  • With a little more work, we can use do.call() to solve the second problem. We first create a list of arguments, then name that, then use do.call():

    args <- list(val)
names(args) <- var

do.call("data.frame", args)
#>   x
#> 1 4
#> 2 3
#> 3 9

Some base functions (including interaction(), expand.grid(), options(), and par()) use a trick to avoid do.call(): if the first component of ... is a list, they’ll take its components instead of looking at the other elements of .... The implementation looks something like this:

f <- function(...) {
  dots <- list(...)
  if (length(dots) == 1 && is.list(dots[[1]])) {
    dots <- dots[[1]]
  }
  
  # Do something
  ...
}

Another approach to avoiding do.call() is found in the RCurl::getURL() function written by Duncan Temple Lang. getURL() takes both ... and .dots which are concatenated together and looks something like this:

f <- function(..., .dots) {
  dots <- c(list(...), .dots)
  # Do something
}

At the time I discovered it, I found this technique particularly compelling so you can see it used throughout the tidyverse. Now, however, I prefer the approach described previously.

19.6.5 Exercises

  1. One way to implement exec() is shown below. Describe how it works. What are the key ideas?

    exec <- function(f, ..., .env = caller_env()) {
  args <- list2(...)
  do.call(f, args, envir = .env)
}

  2. Carefully read the source code for interaction(), expand.grid(), and par(). Compare and contrast the techniques they use for switching between dots and list behaviour.

  3. Explain the problem with this definition of set_attr()

    set_attr <- function(x, ...) {
  attr <- rlang::list2(...)
  attributes(x) <- attr
  x
}
set_attr(1:10, x = 10)
#> Error in attributes(x) <- attr: attributes must be named

  1. This is admittedly not the most creative of names, but it clearly suggests it’s something that has been added to R after the fact.↩︎