step_nzv creates a specification of a recipe step
that will potentially remove variables that are highly sparse
A recipe object. The step will be added to the sequence of operations for this recipe.
One or more selector functions to choose variables
for this step. See
Not used by this step since no new variables are created.
A logical to indicate if the quantities for preprocessing have been estimated.
Numeric parameters for the filtering process. See the Details section below.
A list of options for the filter (see Details below).
A character string that contains the names of
columns that should be removed. These values are not determined
A logical. Should the step be skipped when the
recipe is baked by
A character string that is unique to this step to identify it.
An updated version of
recipe with the new step added to the
sequence of any existing operations.
This step diagnoses predictors that have one unique value (i.e. are zero variance predictors) or predictors that have both of the following characteristics:
they have very few unique values relative to the number of samples and
the ratio of the frequency of the most common value to the frequency of the second most common value is large.
For example, an example of near-zero variance predictor is one that, for 1000 samples, has two distinct values and 999 of them are a single value.
To be flagged, first, the frequency of the most prevalent value
over the second most frequent value (called the "frequency
ratio") must be above
freq_cut. Secondly, the "percent of
unique values," the number of unique values divided by the total
number of samples (times 100), must also be below
In the above example, the frequency ratio is 999 and the unique value percent is 0.2%.
tidy() this step, a tibble with column
terms (the columns
that will be removed) is returned.
library(modeldata) data(biomass) biomass$sparse <- c(1, rep(0, nrow(biomass) - 1)) biomass_tr <- biomass[biomass$dataset == "Training",] biomass_te <- biomass[biomass$dataset == "Testing",] rec <- recipe(HHV ~ carbon + hydrogen + oxygen + nitrogen + sulfur + sparse, data = biomass_tr) nzv_filter <- rec %>% step_nzv(all_predictors()) filter_obj <- prep(nzv_filter, training = biomass_tr) filtered_te <- bake(filter_obj, biomass_te) any(names(filtered_te) == "sparse") #>  FALSE tidy(nzv_filter, number = 1) #> # A tibble: 1 × 2 #> terms id #> <chr> <chr> #> 1 NA nzv_evI1V tidy(filter_obj, number = 1) #> # A tibble: 1 × 2 #> terms id #> <chr> <chr> #> 1 sparse nzv_evI1V