tikzDevice has a new home

Back in February the tikzDevice package became ORPHANED on CRAN. Consequently Kirill Müller and Yihui Xie searched for a new maintainer. When I read about it some time later, we decided that it makes sense for us to step in here. After a brief mail exchange with Yihui Xie the GitHub repository was transfered to our organization and can now be found at https://github.com/daqana/tikzDevice. Meanwhile I have implemented fixes for the existing warning messages from the CRAN checks and uploaded version 0.12, which is currently making its’ way onto CRAN. The next steps will be to work through the existing issues.

What can one do with the tikzDevice? It is a graphics device similar to pdf() or png(). But instead of an image file that might be included in a report as external graphic, it generates files in the TikZ format that makes LaTeX generate the graphic. This enables consistent fonts between text and graphics and TeX’s capabilities for typesetting mathematical equations within graphics. The pdf vignette contains many examples.

One can even use it in a R-markdown document. A document using

header-includes:
  - \\usepackage{tikz}
  - \\usepackage[osf]{mathpazo}

in the YAML header and

# c.f. https://github.com/daqana/tikzDevice/issues/60
library(knitr)
knit_hooks$set(plot = function(x, options) {
  if ("tikz" %in% options$dev && !options$external) {
    hook_plot_tex(x, options)
  } else hook_plot_md(x, options)
})

library(tikzDevice)
options(tikzLatexPackages = c(
  getOption("tikzLatexPackages"),
  "\\usepackage[osf]{mathpazo}",
  "\\usepackage[T1]{fontenc}",
  "\\usepackage{textcomp}"
))

in a setup chunk will use the same fonts for text and graphics when those are created with dev = "tikz". In this example Palatino with text-figures:

Example Palatino with text-figures

(Unit) Testing Shiny apps using testthat

This blog post explains how to test a Shiny app using shinytest and testthat packages. Basic knowledge about Shiny apps and the principle of unit testing using testthat is useful, but not required here.

Example of a Shiny app

The packages shiny (current version: 1.1.0), testthat (2.0.0) and shinytest (1.3.0) are required for the test presented here and may be installed with install.packages().

Below is a minimal example of a Shiny app (app.R) to be be tested. The app has only a single numerical input and a text output. The entered number n is squared and the result is shown as text.

library(shiny)

ui <- fluidRow(title = 'Minimal app',
               numericInput("num_input", "Please insert a number n:", 0),
               textOutput('text_out')
               )

server <- function(input, output, session) {
  result <- reactive(input$num_input^2)
  output$text_out <- renderText(
    paste("The square of the number n is: n² =", result())
    )
}

shinyApp(ui, server)

And this is how the app looks like:

What is shinytest?

The package shinytest provides automatic testing of a Shiny app. Both the “appearance” of the app, as well as its “internal” state during the program flow can be examined. An interactive user interface can be used to create snapshots (more precisely, reference snapshots) as well as a test file. The test file contains the code required for later generation of the snapshots. Each test run creates new snapshots and compares them to the reference snapshots to automatically detect unexpected behavior of the Shiny app. More about the normal workflow with shinytest can be found here. This blog post, however, describes a different approach of testing; Namely testing with shinytest and testtthat combined [*].

[*] Another way of testing uses the function expect_pass (see ?expect_pass), which needs a test file created by shinytest (as well as reference snapshoots) as an input argument. While this allows quick testing, the approach presented here enables more detailed and specific tests.

Testing Shiny apps using shinytest & testthat

shinytest has the class ShinyDriver (see ?ShinyDriver) which opens the Shiny app in a new R-Session as well as an instance of PhantomJS, and connects the two. PhantomJS is a headless web browser that can be operated by JavaScript. The ShinyDriver object is equipped with various methods that enable, among other things, setting/getting values of different variables (inputs or outputs) in the Shiny app. That way, we can assign arbitrary values to the input variables, “manually” (without the usual user interface of the Shiny app), and then get the output variables.

Example of a test

In the following test, the variable num_input is set to 30 and consequently the variable text_out is tested to see if it becomes the string “The square of the number n is: n² = 900”. More on testing with testthat can be found here.

library(shinytest)
library(testthat)

context("Test Shiny app")

# open Shiny app and PhantomJS
app <- ShinyDriver$new("<path to app.R>")

test_that("output is correct", {
  # set num_input to 30
  app$setInputs(num_input = 30)
  # get text_out
  output <- app$getValue(name = "text_out")
  # test
  expect_equal(output, "The square of the number n is: n² = 900")  
})

# stop the Shiny app
app$stop()

Using the expectation functions of the package testthat it is thus easily possible to test the functionalities of the Shiny app. An advantage of this is that when calling devtools::test() both the tests of the Shiny app and other unit tests are taken into account.

Deeper insights – Exported variables and HTML widgets

Within the server function, we can also define new variables (in addition to the usual inputs and outputs) and export them to be “visible” for shinytest and allow for more detailed examination of the app’s workflows. As an example for the Shiny app shown above, we can save a list of all the numbers n entered and export them as a variable inputs_list (please see the code below).

For different HTML widgets the method findElement can be used via app$findElement(xpath ="here the XPATH") using the XPath parameter. For example, if notifications are used with the showNotification() function in the Shiny app, they can be identified with xpath = "//*[@id=\"shiny-notification-panel\"]" and can be tested correspondingly.

Here is how to export a variable in the Shiny app and display notifications using showNotification().

library(shiny)

# same ui as above
ui <- fluidRow(title = 'Minimal app',
               numericInput("num_input", "Please insert a number n:", 0),
               textOutput('text_out')
               )

server <- function(input, output, session) {
  result <- reactive(input$num_input ^ 2)
  output$text_out <- renderText(
    paste("The square of the number n is: n² =", result())
    )
  # initialising the exported list
  inputs_list <- c()
  observeEvent(input$num_input, {
    # new input will be added to inputs_list
    inputs_list <<- c(inputs_list, input$num_input)
    # show notification
    showNotification(HTML(result()), duration = NULL)
    })
  # export inputs_list
  exportTestValues(inputs_list = {inputs_list})
}

shinyApp(ui, server)

For example, a test might look like this:

library(shinytest)
library(testthat)

context("Test Shiny app")

# open Shiny app and PhantomJS
app <- ShinyDriver$new("<path to app.R>")

test_that("inputs_list is exported correctly", {
  # multiple inputs
  app$setInputs(num_input = 1)
  app$setInputs(num_input = 7)
  app$setInputs(num_input = 42)

  # get exported variable inputs_list
  exported_list <- app$getAllValues()$export$inputs_list

  # test (0 was the initial value)
  expect_equal(exported_list, c(0, 1, 7, 42))  
})

test_that("Notifications include correct text", {
  # identify HTML widget with XPath
  popup <- app$findElement(xpath = "//*[@id=\"shiny-notification-panel\"]")

  # test notification text
  testthat::expect_equal(popup$getText(), "×\n0\n×\n1\n×\n49\n×\n1764")
})

# stop the Shiny app
app$stop()

First CRAN release for dqrng

The dqrng package is now available from CRAN. It is possible to install it using

install.packages("dqrng")

Besides this simplified installation the included RNGs have been updated: Xorshift128+ and Xorshift1024* have been removed in favor of the new Xorshiro256+, c.f. http://xoshiro.di.unimi.it/. Using the provided RNGs from R is unchanged:

library(dqrng)
dqRNGkind("Xoroshiro128+")
dqset.seed(42)
dqrunif(5, min = 2, max = 10)
## [1] 4.498747 9.500660 2.457009 5.249195 2.370585
dqrnorm(5, mean = 3, sd = 5)
## [1]  5.748100 -4.368370 -1.231515 -7.552863  8.959892
dqrexp(5, rate = 4)
## [1] 0.002043171 0.261604407 0.132934079 0.048892818 0.549762841

Fast Random Numbers for R with dqrng

If you need only a few truly random numbers you might use dice or atmospheric noise. However, if you need many random numbers you will have to use a pseudo random number generator (RNG). R includes support for different RNGs (c.f. ?Random) and a wide variety of distributions (c.f. ?distributions). The underlying methods have been well tested, but faster methods are available. The dqrng package provides fast random number generators with good statistical properties for usage with R. It combines these RNGs with fast distribution functions to sample from uniform, normal or exponential distributions.

Installation

At the moment dqrng is not on CRAN, but you can install the current version via drat:

if (!requireNamespace("drat", quietly = TRUE)) install.packages("drat")
drat::addRepo("daqana")
install.packages("dqrng")

Usage

Using the provided RNGs from R is deliberately similar to using R’s build-in RNGs:

library(dqrng)
dqRNGkind("Xoroshiro128+")
dqset.seed(42)
dqrunif(5, min = 2, max = 10)
## [1] 4.498747 9.500660 2.457009 5.249195 2.370585
dqrnorm(5, mean = 3, sd = 5)
## [1]  5.748100 -4.368370 -1.231515 -7.552863  8.959892
dqrexp(5, rate = 4)
## [1] 0.002043171 0.261604407 0.132934079 0.048892818 0.549762841

They are quite a bit faster, though, as we can see by comparing 10 million random draws from different distributions:

N <- 1e7
tm <- microbenchmark(
  runif = runif(N),
  dqrunif = dqrunif(N),
  rnorm = rnorm(N), 
  dqrnorm = dqrnorm(N),
  rexp = rexp(N),
  dqrexp = dqrexp(N))
expr min lq mean median uq max neval
runif 248.16730 251.83371 262.20559 260.33073 265.69415 322.15771 100
dqrunif 34.77413 35.44569 39.40738 36.82459 38.42524 109.96758 100
rnorm 587.40975 596.92850 618.79356 613.08345 624.31043 706.79528 100
dqrnorm 63.17649 64.43796 68.77696 66.80184 68.39577 141.97466 100
rexp 392.79228 397.48715 413.66996 411.14180 420.42473 494.49631 100
dqrexp 52.75875 53.64510 57.15006 55.80021 58.65553 79.11577 100

plot of chunk unnamed-chunk-4

For r* the default Mersenne-Twister was used, while dqr* used Xoroshiro128+ in this comparison. For rnorm the default inversion method was used, while dqrnorm (and dqrexp) used the Ziggurat algorithm from Boost.Random with additional tuning.

Both the RNGs and the distribution functions are distributed as C++ header-only library. See the included vignette for possible usage from C++.

Supported Random Number Generators

Support for the following 64 bit RNGs is currently included:

  • Mersenne-Twister
    The 64 bit variant of the well-known Mersenne-Twister, which is also used as default. This is a conservative default that allows you to take advantage of the fast distribution functions provided by dqrng while staying close to R’s default RNG (32 bit Mersenne-Twister).
  • pcg64
    The default 64 bit variant from the PCG family developed by Melissa O’Neill. See http://www.pcg-random.org for more details.
  • Xoroshiro128+, Xorshift128+, and Xorshift1024*
    RNGs mainly developed by Sebastiano Vigna. They are used as default RNGs in Erlang and different JavaScript engines. See http://xoroshiro.di.unimi.it/ for more details.

RcppArrayFire 0.0.2: Rcpp integration for ArrayFire

The RcppArrayFire package uses Rcpp to provide an interface from R to and from the ArrayFire library, an open source library that can make use of GPUs and other hardware accelerators via CUDA or OpenCL.

The official R bindings expose ArrayFire data structures as objects in R, which would require a large amount of code to support all the methods defined in ArrayFire’s C/C++ API. RcppArrayFire instead, which is derived from RcppFire by Kazuki Fukui, follows the lead of packages like RcppArmadillo or RcppEigen to provide seamless communication between R and ArrayFire at the C++ level.

Installation

Please note that currently RcppArrayFire has only been tested on Linux systems.

Prerequisites

In order to use RcppArrayFire you will need development tools for R, Rcpp and the ArrayFire library and header files. On a sufficiently recent Debian based or derived system, this can be achieved with:

sudo apt-get install r-base-dev r-cran-rcpp libarrayfire-dev libarrayfire-unified3 

This will install the unified and CPU backends. The CUDA backend has not been packaged for Debian, and usage of the packaged OpenCL backend is hindered by a bug in the clBLAS package. For serious usage it is currently better to build from source or use the binary installer from ArrayFire:

sudo apt-get install r-base-dev r-cran-rcpp libglfw3
# download installer from http://arrayfire.com/download/
sudo sh arrayfire-installer.sh --exclude-subdir --prefix=/usr/local

In the last command you have to adjust the name of the installer script and (optionally) the installation prefix. For GPU support, you have to install additional drivers. For many build-in Intel GPUs, you can use

sudo apt-get install beignet-opencl-icd ocl-icd-libopencl1

Installing CUDA or other OpenCL drivers is beyond the scope of this post, but see the ArrayFire documentation for details.

Package installation

RcppArrayFire is not on CRAN, but you can install the current version via drat:

#install.packages("drat")
drat::addRepo("daqana")
install.packages("RcppArrayFire")

If you have installed ArrayFire in a non-standard directory, you have to use the configure argument --with-arrayfire:

install.packages("RcppArrayFire", configure.args = "--with-arrayfire=/opt/arrayfire-3")

Usage

Calculating pi by simulation

Let’s look at the classical example of calculating pi via simulation. The basic idea is to generate a large number of random points within the unit square. An approximation for pi can then be calculated from the ratio of points within the unit circle to the total number of points. A vectorized implementation in R might look like this:

piR <- function(N) {
    x <- runif(N)
    y <- runif(N)
    4 * sum(sqrt(x^2 + y^2) < 1.0) / N
}

set.seed(42)
system.time(cat("pi ~= ", piR(10^7), "\n"))
## pi ~=  3.140899
##    user  system elapsed 
##   0.688   0.076   0.766

A simple way to use C++ code in R is to use the inline package or cppFunction() from Rcpp, which are both possible with RcppArrayFire. An implementation in C++ using ArrayFire might look like this:

src <- '
double piAF (const int N) {
    array x = randu(N, f32);
    array y = randu(N, f32);
    return 4.0 * sum<float>(sqrt(x*x + y*y) < 1.0) / N;
}'
Rcpp::cppFunction(code = src, depends = "RcppArrayFire", includes = "using namespace af;")

RcppArrayFire::arrayfire_set_seed(42)
system.time(cat("pi ~= ", piAF(10^7), "\n"))
## pi ~=  3.141066
##    user  system elapsed 
##   0.036   0.024  0.019

Several things are worth noting:

(1) The syntax is almost identical. Besides the need for using types and a different function name when generating random numbers, the argument f32 to randu as well as the float type catches the eye. These instruct ArrayFire to use single precision floats, since not all devices support double precision floating point numbers. If you want to use double precision, you have to specify f64 and double.

(2) The results are not the same, since ArrayFire uses a different random number generator.

(3) The speed-up is quite impressive. However, sometimes the first invocation of a function is not as fast as expected due to the just-in-time compilation used by ArrayFire.

Arrays as parameters

Up to now we have only considered simple types like double or int as function parameters and return values. However, we can also use arrays. Consider the matrix product X’ X for a random matrix X in R:

set.seed(42)
N <- 40
X <- matrix(rnorm(N * N * 2), ncol = N)
tXXR <- t(X) %*% X

The matrix multiplication can be implemented with RcppArrayFire using the appropriate matmul function:

src <- '
af::array squareMatrix(const RcppArrayFire::typed_array<f32>& x) {
    return af::matmulTN(x ,x);
}'
Rcpp::cppFunction(code = src, depends = "RcppArrayFire")
tXXGPU <- squareMatrix(X)

all.equal(tXXR, tXXGPU)
## [1] "Mean relative difference: 1.372856e-07"

Since an object of type af::array can contain different data types, the templated wrapper class RcppArrayFire::typed_array<> is used to indicate the desired data type when converting from R to C++. Again single precision floats are used with ArrayFire, which explains the difference between the two results. We can be sure that double precision is supported by switching the computation backend to “CPU”, which produces identical results:

src <- '
af::array squareMatrixF64(const RcppArrayFire::typed_array<f64>& x) {
    return af::matmulTN(x ,x);
}'
Rcpp::cppFunction(code = src, depends = "RcppArrayFire")

RcppArrayFire::arrayfire_set_backend("CPU")
tXXCPU <- squareMatrixF64(X)
RcppArrayFire::arrayfire_set_backend("DEFAULT")

all.equal(tXXR, tXXCPU)
## [1] TRUE

Usage in a package

More serious functions should be defined in a permanent fashion. To facilitate this, RcppArrayFire contains the function RcppArraFire.package.skeleton(). This functions initialises a package with suitable configure script for linking with ArrayFire and RcppArrayFire. In order to implement new functionality you can then write C++ functions and save them in the src folder. Functions that should be callable from R should be marked with the [[Rcpp::export]] attribute. See the Rcpp vignettes on attributes and package writing for further details.

Outlook

Besides testing the package on other platforms than Linux, future versions might include