Chapter 11 Namespace

The package namespace (as recorded in the NAMESPACE file) is one of the more confusing parts of building a package. It’s a fairly advanced topic, and by-and-large, not that important if you’re only developing packages for yourself. However, understanding namespaces is vital if you plan to submit your package to CRAN. This is because CRAN requires that your package plays nicely with other packages.

When you first start using namespaces, it’ll seem like a lot of work for little gain. However, having a high quality namespace helps encapsulate your package and makes it self-contained. This ensures that other packages won’t interfere with your code, that your code won’t interfere with other packages, and that your package works regardless of the environment in which it’s run.

11.1 Motivation

As the name suggests, namespaces provide “spaces” for “names”. They provide a context for looking up the value of an object associated with a name.

Without knowing it, you’ve probably already used namespaces. For example, have you ever used the :: operator? It disambiguates functions with the same name. For example, both plyr and Hmisc provide a summarize() function. If you load plyr, then Hmisc, summarize() will refer to the Hmisc version. But if you load the packages in the opposite order, summarize() will refer to the plyr version. This can be confusing. Instead, you can explicitly refer to specific functions: Hmisc::summarize() and plyr::summarize(). Then the order in which the packages are loaded won’t matter.

Namespaces make your packages self-contained in two ways: the imports and the exports. The imports defines how a function in one package finds a function in another. To illustrate, consider what happens when someone changes the definition of a function that you rely on: for example, the simple nrow() function in base R:

nrow
#> function (x) 
#> dim(x)[1L]
#> <bytecode: 0x563571579980>
#> <environment: namespace:base>

It’s defined in terms of dim(). So what will happen if we override dim() with our own definition? Does nrow() break?

dim <- function(x) c(1, 1)
dim(mtcars)
#> [1] 1 1
nrow(mtcars)
#> [1] 32

Surprisingly, it does not! That’s because when nrow() looks for an object called dim(), it uses the package namespace, so it finds dim() in the base environment, not the dim() we created in the global environment.

The exports helps you avoid conflicts with other packages by specifying which functions are available outside of your package (internal functions are available only within your package and can’t easily be used by another package). Generally, you want to export a minimal set of functions; the fewer you export, the smaller the chance of a conflict. While conflicts aren’t the end of the world because you can always use :: to disambiguate, they’re best avoided where possible because it makes the lives of your users easier.

11.2 Search path

To understand why namespaces are important, you need a solid understanding of search paths. To call a function, R first has to find it. R does this by first looking in the global environment. If R doesn’t find it there, it looks in the search path, the list of all the packages you have attached. You can see this list by running search(). For example, here’s the search path for the code in this book:

search()
#> [1] ".GlobalEnv"        "package:stats"     "package:graphics" 
#> [4] "package:grDevices" "package:utils"     "package:datasets" 
#> [7] "package:methods"   "Autoloads"         "package:base"

There’s an important difference between loading and attaching a package. Normally when you talk about loading a package you think of library(), but that actually attaches the package.

If a package is installed,

  • Loading will load code, data and any DLLs; register S3 and S4 methods; and run the .onLoad() function. After loading, the package is available in memory, but because it’s not in the search path, you won’t be able to access its components without using ::. Confusingly, :: will also load a package automatically if it isn’t already loaded. It’s rare to load a package explicitly, but you can do so with requireNamespace() or loadNamespace().

  • Attaching puts the package in the search path. You can’t attach a package without first loading it, so both library() or require() load then attach the package. You can see the currently attached packages with search().

If a package isn’t installed, loading (and hence attaching) will fail with an error.

To see the differences more clearly, consider two ways of running expect_that() from the testthat package. If we use library(), testthat is attached to the search path. If we use ::, it’s not.

old <- search()
testthat::expect_equal(1, 1)
setdiff(search(), old)
#> character(0)
expect_true(TRUE)
#> Error in expect_true(TRUE): could not find function "expect_true"
    
library(testthat)
expect_equal(1, 1)
setdiff(search(), old)
#> [1] "package:testthat"
expect_true(TRUE)

There are four functions that make a package available. They differ based on whether they load or attach, and what happens if the package is not found (i.e., throws an error or returns FALSE).

Throws error Returns FALSE
Load loadNamespace("x") requireNamespace("x", quietly = TRUE)
Attach library(x) require(x, quietly = TRUE)

Of the four, you should only ever use two:

  • Use library(x) in data analysis scripts. It will throw an error if the package is not installed, and will terminate the script. You want to attach the package to save typing. Never use library() in a package.

  • Use requireNamespace("x", quietly = TRUE) inside a package if you want a specific action (e.g. throw an error) depending on whether or not a suggested package is installed.

You never need to use require() (requireNamespace() is almost always better), or loadNamespace() (which is only needed for internal R code). You should never use require() or library() in a package: instead, use the Depends or Imports fields in the DESCRIPTION.

Now’s a good time to come back to an important issue which we glossed over earlier. What’s the difference between Depends and Imports in the DESCRIPTION? When should you use one or the other?

Listing a package in either Depends or Imports ensures that it’s installed when needed. The main difference is that where Imports just loads the package, Depends attaches it. There are no other differences. The rest of the advice in this chapter applies whether or not the package is in Depends or Imports.

Unless there is a good reason otherwise, you should always list packages in Imports not Depends. That’s because a good package is self-contained, and minimises changes to the global environment (including the search path). The only exception is if your package is designed to be used in conjunction with another package. For example, the analogue package builds on top of vegan. It’s not useful without vegan, so it has vegan in Depends instead of Imports. Similarly, ggplot2 should really Depend on scales, rather than Importing it.

Now that you understand the importance of the namespace, let’s dive into the nitty gritty details. The two sides of the package namespace, imports and exports, are both described by the NAMESPACE. You’ll learn what this file looks like in the next section. In the section after that, you’ll learn the details of exporting and importing functions and other objects.

11.3 The NAMESPACE

The following code is an excerpt of the NAMESPACE file from the testthat package.

# Generated by roxygen2 (4.0.2): do not edit by hand
S3method(as.character,expectation)
S3method(compare,character)
export(auto_test)
export(auto_test_package)
export(colourise)
export(context)
exportClasses(ListReporter)
exportClasses(MinimalReporter)
importFrom(methods,setRefClass)
useDynLib(testthat,duplicate_)
useDynLib(testthat,reassign_function)

You can see that the NAMESPACE file looks a bit like R code. Each line contains a directive: S3method(), export(), exportClasses(), and so on. Each directive describes an R object, and says whether it’s exported from this package to be used by others, or it’s imported from another package to be used locally.

In total, there are eight namespace directives. Four describe exports:

  • export(): export functions (including S3 and S4 generics).
  • exportPattern(): export all functions that match a pattern.
  • exportClasses(), exportMethods(): export S4 classes and methods.
  • S3method(): export S3 methods.

And four describe imports:

  • import(): import all functions from a package.
  • importFrom(): import selected functions (including S4 generics).
  • importClassesFrom(), importMethodsFrom(): import S4 classes and methods.
  • useDynLib(): import a function from C. This is described in more detail in compiled code.

I don’t recommend writing these directives by hand. Instead, in this chapter you’ll learn how to generate the NAMESPACE file with roxygen2. There are three main advantages to using roxygen2:

  • Namespace definitions live next to its associated function, so when you read the code it’s easier to see what’s being imported and exported.

  • Roxygen2 abstracts away some of the details of NAMESPACE. You only need to learn one tag, @export, which will automatically generate the right directive for functions, S3 methods, S4 methods and S4 classes.

  • Roxygen2 makes NAMESPACE tidy. No matter how many times you use @importFrom foo bar you’ll only get one importFrom(foo, bar) in your NAMESPACE. This makes it easy to attach import directives to every function that need them, rather than trying to manage in one central place.

Note that you can choose to use roxygen2 to generate just NAMESPACE, just man/*.Rd, or both. If you don’t use any namespace related tags, roxygen2 won’t touch NAMESPACE. If you don’t use any documentation related tags, roxygen2 won’t touch man/.

11.4 Workflow

Generating the namespace with roxygen2 is just like generating function documentation with roxygen2. You use roxygen2 blocks (starting with #') and tags (starting with @). The workflow is the same:

  1. Add roxygen comments to your .R files.

  2. Run devtools::document() (or press Ctrl/Cmd + Shift + D in RStudio) to convert roxygen comments to .Rd files.

  3. Look at NAMESPACE and run tests to check that the specification is correct.

  4. Rinse and repeat until the correct functions are exported.

11.5 Exports

For a function to be usable outside of your package, you must export it. When you create a new package with usethis::create_package(), it produces a temporary NAMESPACE that exports everything in your package that doesn’t start with . (a single period). If you’re just working locally, it’s fine to export everything in your package. However, if you’re planning on sharing your package with others, it’s a really good idea to only export needed functions. This reduces the chances of a conflict with another package.

To export an object, put @export in its roxygen block. For example:

#' @export
foo <- function(x, y, z) {
  ...
}

This will then generate export(), exportMethods(), exportClass() or S3method() depending on the type of the object.

You export functions that you want other people to use. Exported functions must be documented, and you must be cautious when changing their interface — other people are using them! Generally, it’s better to export too little than too much. It’s easy to export things that you didn’t before; it’s hard to stop exporting a function because it might break existing code. Always err on the side of caution, and simplicity. It’s easier to give people more functionality than it is to take away stuff they’re used to.

I believe that packages that have a wide audience should strive to do one thing and do it well. All functions in a package should be related to a single problem (or a set of closely related problems). Any functions not related to that purpose should not be exported. For example, most of my packages have a utils.R file that contains many small functions that are useful for me, but aren’t part of the core purpose of those packages. I never export these functions.

# Defaults for NULL values
`%||%` <- function(a, b) if (is.null(a)) b else a

# Remove NULLs from a list
compact <- function(x) {
  x[!vapply(x, is.null, logical(1))]
}

That said, if you’re creating a package for yourself, it’s far less important to be so disciplined. Because you know what’s in your package, it’s fine to have a local “misc” package that contains a passel of functions that you find useful. But I don’t think you should release such a package.

The following sections describe what you should export if you’re using S3, S4 or RC.

11.5.1 S3

If you want others to be able to create instances of an S3 class, @export the constructor function. S3 generics are just regular R functions, you can @export them like functions.

S3 methods represent the most complicated case because there are four different scenarios:

  • A method for an exported generic: export every method.

  • A method for an internal generic: technically, you don’t need to export these methods. However, I recommend exporting every S3 method you write because it’s simpler and makes it less likely that you’ll introduce hard to find bugs. Use devtools::missing_s3() to list all S3 methods that you’ve forgotten to export.

  • A method for a generic in a required package. You’ll need to import the generic (see below), and export the method.

  • A method for a generic in a suggested package. Namespace directives must refer to available functions, so they can not reference suggested packages. It’s possible to use package hooks and code to add this at run-time, but this is sufficiently complicated that I currently wouldn’t recommend it. Instead, you’ll have to design your package dependencies in a way that avoids this scenario.

11.5.2 S4

S4 classes: if you want others to be able to extend your class, @export it. If you want others to create instances of your class but not to extend it, @export the constructor function, not the class.

# Can extend and create with new("A", ...)
#' @export
setClass("A")

# Can extend and create with new("B", ...). You can use B()
# to construct instances in your own code, but others can not
#' @export
B <- setClass("B")

# Can create with C(...) and new("C", ...), but can't create
# a subclass that extends C
#' @export C
C <- setClass("C")

# Can extend and create with D(...) or new("D", ...)
#' @export D
#' @exportClass D
D <- setClass("D")

S4 generics: @export if you want the generic to be publicly usable.

S4 methods: you only need to @export methods for generics that you did not define. But I think it’s a good idea to @export every method: that way you don’t need to remember whether or not you created the generic.

11.5.3 RC

The principles used for S4 classes apply here. Note that due to the way that RC is currently implemented, it’s typically impossible for your classes to be extended outside of your package.

11.5.4 Data

As you’ll learn about in data, files that live in data/ don’t use the usual namespace mechanism and don’t need to be exported.

11.6 Imports

NAMESPACE also controls which external functions can be used by your package without having to use ::.

It’s confusing that both DESCRIPTION (through the Imports field) and NAMESPACE (through import directives) seem to be involved in imports. This is just an unfortunate choice of names. The Imports field really has nothing to do with functions imported into the namespace: it just makes sure the package is installed when your package is. It doesn’t make functions available. You need to import functions in exactly the same way regardless of whether or not the package is attached.

Depends is just a convenience for the user: if your package is attached, it also attaches all packages listed in Depends. If your package is loaded, packages in Depends are loaded, but not attached, so you need to qualify function names with :: or specifically import them.

It’s common for packages to be listed in Imports in DESCRIPTION, but not in NAMESPACE. In fact, this is what I recommend: list the package in DESCRIPTION so that it’s installed, then always refer to it explicitly with pkg::fun(). Unless there is a strong reason not to, it’s better to be explicit. It’s a little more work to write, but a lot easier to read when you come back to the code in the future. The converse is not true. Every package mentioned in NAMESPACE must also be present in the Imports or Depends fields.

11.6.1 R functions

If you are using just a few functions from another package, my recommendation is to note the package name in the Imports: field of the DESCRIPTION file and call the function(s) explicitly using ::, e.g., pkg::fun().

If you are using functions repeatedly, you can avoid :: by importing the function with @importFrom pkg fun. This also has a small performance benefit, because :: adds approximately 5 µs to function evaluation time. Operators can also be imported in a similar manner, e.g., @importFrom magrittr %>%.

Alternatively, if you are repeatedly using many functions from another package, you can import all of them using @import package. This is the least recommended solution because it makes your code harder to read (you can’t tell where a function is coming from), and if you @import many packages, it increases the chance of conflicting function names.

11.6.2 S3

S3 generics are just functions, so the same rules for functions apply. S3 methods always accompany the generic, so as long as you can access the generic (either implicitly or explicitly), the methods will also be available. In other words, you don’t need to do anything special for S3 methods. As long as you’ve imported the generic, all the methods will also be available.

11.6.3 S4

To use classes defined in another package, place @importClassesFrom package ClassA ClassB ... next to the classes that inherit from the imported classes, or next to the methods that implement a generic for the imported classes.

To use generics defined in another package, place @importMethodsFrom package GenericA GenericB ... next to the methods that use the imported generics.

Since S4 is implemented in the methods package, you need to make sure it’s available. This is easy to overlook because while the methods package is always available in the search path when you’re working interactively, it’s not automatically loaded by Rscript, the tool often used to run R from the command line.

  • Pre R 3.2.0: Depends: methods in DESCRIPTION.
    Post R 3.2.0: Imports: methods in DESCRIPTION.

  • Since you’ll being using a lot of functions from methods, you’ll probably also want to import the complete package with:

    #' @import methods
NULL

    Or you might just want to import the most commonly used functions:

    #' @importFrom methods setClass setGeneric setMethod setRefClass
NULL

    Here I’m documenting “NULL” to make it clear that these directives don’t apply to just one function. It doesn’t matter where they go, but if you have package docs, as described in documenting packages, that’s a natural place to put them.

11.6.4 Compiled functions

To make C/C++ functions available in R, see compiled code.