Utilizing quosures to create ultra flexible filtering controls in R Shiny

I build R Shiny apps quite a lot, and one of the common uses is to allow dynamic filtering of the underlying data, so that you can adjust a chart or a table based on some particular subset of interest.

Normally, we hardcode the filtering options into a sidebar menu of some form, and define input picklists for the fields of interest. For example, if we were filtering the mtcars dataset, we might offer an input picklist like this on the UI side:

shiny::selectInput('cyl', "Select no of cylinders", choices = mtcars$cyl %>% unique() %>% sort())

and then we might filter mtcars using input$cyl on the server side, for example:

df <- shiny::reactive({
mtcars %>% dplyr::filter(cyl == input$cyl)
})

I’ve been working on creating a more general structure which allows a fully flexible, dynamic filtering system (akin to filtering in a MSExcel spreadsheet), allowing the user to select whatever field they wish to filter on, and whatever values for that field that they are interested in. I also wanted to set it up so that it could be applied generically to any dataset, requiring the user only to specify the dataset and the subset of column names on which filtering is allowed.

Referential transparency of Shiny inputs

In order to do this, we have to get around non-standard evaluation issues. In the mtcars example above, cylis treated differently frominput$cyl. cyl is referentially transparent in the context of mtcars, but input$cyl is passed as an input variable and its value is quoted. So, for example, if we select input$cyl = 6, this will be interpreted as we would expect in dplyr:

df <- mtcars %>% dplyr::filter(cyl == '6')

So this works fine, but what if we wanted to select the column as a shiny input also? For example, you may want your user to be able to select from 20 or 30 different columns to filter on. Maybe we can create an input$col for the selected column to filter on and an input$val for the value selected, as follows:

df <- shiny::reactive({
mtcars %>% dplyr::filter(input$col == input$val)
})

Well, no, we can’t because input$col is passed as a quoted value and is not referentially transparent.

This is where quosures can be really useful, and can allow you to abstract out your filtering capability (and your grouping, selecting and all those other lovely things that we enjoy about dplyr).

Consider setting up the following simple function to allow you to choose both the column and the value you wish to filter on:

filter1_by <- function(df, fcol1, fv1) {

filter_var1 <- dplyr::quo(fcol1)
  df %>% filter_at(vars(!!filter_var1), all_vars(. == fv1))
}

This simple little function allows us to treat the column choice ( fcol1) separately from the value choice ( fval1) using filter_at(). fcol1 is now captured as a quoted expression along with its environment, making it referentially transparent, and then we simply use the !! (bangbang) operator to strip the quotes away so dplyr can accept the column name unquoted but referentially transparent.

Needless to say this can be expanded to filter on any number of inputs selected from any number of columns. A function to allow three filters, for example, would look like:

filter3_by <- function(df, fcol1, fv1, fcol2, fv2, fcol3, fv3) {
filter_var1 <- dplyr::quo(fcol1)
filter_var2 <- dplyr::quo(fcol2)
filter_var3 <- dplyr::quo(fcol3)
df %>% 
filter_at(vars(!!filter_var1), all_vars(. == fv1)) %>%
filter_at(vars(!!filter_var2), all_vars(. == fv2)) %>%
filter_at(vars(!!filter_var3), all_vars(. == fv3))
)

Let’s build a full example

Let’s proceed to build a full example using mtcars. Let’s say that we want the user to be able to pick up to three columns to filter on, from any of the available columns in mtcars.

We first create a vector from the column names of mtcarswhich we will use as our initial input choice:

fields <- colnames(mtcars)

Let’s set up three functions to filter based on one, two, or three columns:

# filter on 1 columns
filter1_by <- function(df, fcol1, fv1) {
filter_var1 <- dplyr::quo(fcol1)
df %>% 
filter_at(vars(!!filter_var1), all_vars(. == fv1))
)
# filter on 2 columns
filter2_by <- function(df, fcol1, fv1, fcol2, fv2) {
filter_var1 <- dplyr::quo(fcol1)
filter_var2 <- dplyr::quo(fcol2)
df %>% 
filter_at(vars(!!filter_var1), all_vars(. == fv1)) %>%
filter_at(vars(!!filter_var2), all_vars(. == fv2))
)
# filter on 3 columns
filter3_by <- function(df, fcol1, fv1, fcol2, fv2, fcol3, fv3) {
filter_var1 <- dplyr::quo(fcol1)
filter_var2 <- dplyr::quo(fcol2)
filter_var3 <- dplyr::quo(fcol3)
df %>% 
filter_at(vars(!!filter_var1), all_vars(. == fv1)) %>%
filter_at(vars(!!filter_var2), all_vars(. == fv2)) %>%
filter_at(vars(!!filter_var3), all_vars(. == fv3))
)

Now let’s build a structure of Shiny inputs allowing us filter on at least one column, but up to three columns, using checkboxes to allow the user to decide if they want additional filters:

shiny::mainPanel(
# select first filter column from fields vector
shiny::selectInput("filter1", "Select filter column 1:",
choices = fields),
# reference a uiOutput that will offer values for first column
shiny::uiOutput("filter1choice"),
# offer a checkbox to allow user to select a second filter
shiny::checkboxInput("filter2req", "Add second filter?"),
# set further conditional panels to appear in the same fashion
shiny::conditionalPanel(condition = 'input.filter2req',
shiny::uiOutput("filter2eval"),
shiny::uiOutput("filter2choice"),
shiny::checkboxInput("filter3req",
"Add third filter?")),
shiny::conditionalPanel(condition = 'input.filter3req &
input.filter2req',
shiny::uiOutput("filter3eval"),
shiny::uiOutput("filter3choice"))

)

Now we need to build uiOutputs that populate according to the selected filter columns, and are repsonsive to what has already been selected:

# vector of picklist values for the first selected filter 
choicevec1 <- reactive({
mtcars %>% dplyr::select(input$filter1) %>% unique() %>% dplyr::arrange_(input$filter1)
})
# renders the picklist for the first selected filter
output$filter1choice <- renderUI(
selectizeInput("filter1val", "Select filter 1 condition:", choices = choicevec1(), multiple = TRUE)
)
# second column chosen from all remaining fields
output$filter2eval <- renderUI({
selectInput("filter2", "Select filter criteria 2:", choices = sort(fields[fields != input$filter1]))
})
# vector of picklist values for the second selected filter
choicevec2 <- reactive({
filter1_by(mtcars, input$filter1, input$filter1val) %>%
dplyr::select(input$filter2) %>%
unique() %>%
dplyr::arrange_(input$filter2)
})
# renders picklist for filter 2
output$filter2choice <- renderUI(
selectizeInput("filter2val", "Select filter 2 condition:", choices = choicevec2(), multiple = TRUE)
)
# third column selected from remaining fields
output$filter3eval <- renderUI({
selectInput("filter3", "Select filter criteria 3:", choices = sort(fields[!fields %in% c(input$filter1, input$filter2)]))
})
# vector of picklist values for third selected column
choicevec3 <- reactive({
filter2_by(mtcars, input$filter1, input$filter1val,
input$filter2, input$filter2val) %>%
dplyr::select(input$filter3) %>%
unique() %>%
dplyr::arrange_(input$filter3)
})
# render picklist for filter 3
output$filter3choice <- renderUI(
selectizeInput("filter3val", "Select filter 3 condition:", choices = choicevec3(), multiple = TRUE)
)

So this is all we need to do on the UI side.

On the server side, we now just need to define the filter we need based on the inputs selected:

filtered_mtcars <- reactive({
  # case when all three filters are used
if (input$filter3req & input$filter2req) {
filter3_by(mtcars, input$filter1, input$filter1val,
input$filter2, input$filter2val,
input$filter3, input$filter3val)
} else if (input$filter2req) {
# case when two filters are used
filter2_by(mtcars, input$filter1, input$filter1val,
input$filter2, input$filter2val)
} else {
# case when only one filter is used
filter1_by(mtcars, input$filter1, input$filter1val)
}
})

Now you can just display your reactive dataframe filtered_mtcars()or do manipulations on it.

Using this method in practice and further development

These functions are easily liftable into any app you are designing which needs this kind of flexible filtering. You can easily expand to more filter selections by simply writing further functions filter4_by, filter5_by, etc. You can also easily adjust to allow multiple value selection in picklists by using selectizeInput() and replacing == with %in% inside filter_at().

See here for a really simple demonstration app I constructed showing medal tallies at the Olympics which allows up to three filters. The Github code for this is here.

Can you help improve this? Rather than writing all these separate functions filter1_by, filter2_by, etc, ideally we would only need a single filter function as follows:

filter_by <- function (df, ...) {
filter_conditions <- quos(...)
df %>% dplyr::filter(!!!filter_conditions)
}

where the inputs are lists of conditions. These inputs will need some sort of tweaking to cope with non-standard evaluation, but I haven’t yet worked out how they should be phrased.

Let me know if you crack this.

Leave a Reply

%d bloggers like this: