Package 'mlr3torch'

Description

Deep Learning library that extends the mlr3 framework by building upon the 'torch' package. It allows to conveniently build, train, and evaluate deep learning models without having to worry about low level details. Custom architectures can be created using the graph language defined in 'mlr3pipelines'.

Options

• mlr3torch.cache: Whether to cache the downloaded data (TRUE) or not (FALSE, default). This can also be set to a specific folder on the file system to be used as the cache directory.

Author(s)

Maintainer: Sebastian Fischer [email protected] (ORCID)

Authors:

• Martin Binder [email protected]

Other contributors:

• Bernd Bischl [email protected] (ORCID) [contributor]

• Lukas Burk [email protected] (ORCID) [contributor]

• Florian Pfisterer [email protected] (ORCID) [contributor]

• Carson Zhang [email protected] [contributor]

See Also

Useful links:

• https://mlr3torch.mlr-org.com/

• https://github.com/mlr-org/mlr3torch/

• Report bugs at https://github.com/mlr-org/mlr3torch/issues

Description

Converts the input to a DataDescriptor.

Usage

as_data_descriptor(x, dataset_shapes, ...)

Arguments

Examples

ds = dataset("example",
  initialize = function() self$iris = iris[, -5],
  .getitem = function(i) list(x = torch_tensor(as.numeric(self$iris[i, ]))),
  .length = function() nrow(self$iris)
)()
as_data_descriptor(ds, list(x = c(NA, 4L)))

# if the dataset has a .getbatch method, the shapes are inferred
ds2 = dataset("example",
  initialize = function() self$iris = iris[, -5],
  .getbatch = function(i) list(x = torch_tensor(as.matrix(self$iris[i, ]))),
  .length = function() nrow(self$iris)
)()
as_data_descriptor(ds2)

Description

Convert a object to a lazy_tensor.

Usage

as_lazy_tensor(x, ...)

## S3 method for class 'dataset'
as_lazy_tensor(x, dataset_shapes = NULL, ids = NULL, ...)

Arguments

Examples

iris_ds = dataset("iris",
  initialize = function() {
    self$iris = iris[, -5]
  },
  .getbatch = function(i) {
    list(x = torch_tensor(as.matrix(self$iris[i, ])))
  },
  .length = function() nrow(self$iris)
)()
# no need to specify the dataset shapes as they can be inferred from the .getbatch method
# only first 5 observations
as_lazy_tensor(iris_ds, ids = 1:5)
# all observations
head(as_lazy_tensor(iris_ds))

iris_ds2 = dataset("iris",
  initialize = function() self$iris = iris[, -5],
  .getitem = function(i) list(x = torch_tensor(as.numeric(self$iris[i, ]))),
  .length = function() nrow(self$iris)
)()
# if .getitem is implemented we cannot infer the shapes as they might vary,
# so we have to annotate them explicitly
as_lazy_tensor(iris_ds2, dataset_shapes = list(x = c(NA, 4L)))[1:5]

# Convert a matrix
lt = as_lazy_tensor(matrix(rnorm(100), nrow = 20))
materialize(lt[1:5], rbind = TRUE)

Description

Convert a torch scheduler generator to a CallbackSetLRScheduler.

Usage

as_lr_scheduler(x, step_on_epoch)

Arguments

Description

Converts an object to a TorchCallback.

Usage

as_torch_callback(x, clone = FALSE, ...)

Arguments

Value

TorchCallback.

See Also

Other Callback: TorchCallback, as_torch_callbacks(), callback_set(), mlr3torch_callbacks, mlr_callback_set, mlr_callback_set.checkpoint, mlr_callback_set.progress, mlr_callback_set.tb, mlr_callback_set.unfreeze, mlr_context_torch, t_clbk(), torch_callback()

Description

Converts an object to a list of TorchCallback.

Usage

as_torch_callbacks(x, clone, ...)

Arguments

Value

list() of TorchCallbacks

See Also

Other Callback: TorchCallback, as_torch_callback(), callback_set(), mlr3torch_callbacks, mlr_callback_set, mlr_callback_set.checkpoint, mlr_callback_set.progress, mlr_callback_set.tb, mlr_callback_set.unfreeze, mlr_context_torch, t_clbk(), torch_callback()

Other Torch Descriptor: TorchCallback, TorchDescriptor, TorchLoss, TorchOptimizer, as_torch_loss(), as_torch_optimizer(), mlr3torch_losses, mlr3torch_optimizers, t_clbk(), t_loss(), t_opt()

Description

Converts an object to a TorchLoss.

Usage

as_torch_loss(x, clone = FALSE, ...)

Arguments

Value

TorchLoss.

See Also

Other Torch Descriptor: TorchCallback, TorchDescriptor, TorchLoss, TorchOptimizer, as_torch_callbacks(), as_torch_optimizer(), mlr3torch_losses, mlr3torch_optimizers, t_clbk(), t_loss(), t_opt()

Description

Converts an object to a TorchOptimizer.

Usage

as_torch_optimizer(x, clone = FALSE, ...)

Arguments

Value

TorchOptimizer

See Also

Other Torch Descriptor: TorchCallback, TorchDescriptor, TorchLoss, TorchOptimizer, as_torch_callbacks(), as_torch_loss(), mlr3torch_losses, mlr3torch_optimizers, t_clbk(), t_loss(), t_opt()

Description

Asserts whether something is a lazy tensor.

Usage

assert_lazy_tensor(x)

Arguments

Description

First tries cuda, then cpu.

Usage

auto_device(device = NULL)

Arguments

Description

Converts a data frame of categorical data into a long tensor by converting the data to integers. No input checks are performed.

Usage

batchgetter_categ(data, ...)

Arguments

Description

Converts a data frame of numeric data into a float tensor by calling as.matrix(). No input checks are performed

Usage

batchgetter_num(data, ...)

Arguments

Description

Creates an R6ClassGenerator inheriting from CallbackSet. Additionally performs checks such as that the stages are not accidentally misspelled. To create a TorchCallback use torch_callback().

In order for the resulting class to be cloneable, the private method ⁠$deep_clone()⁠ must be provided.

Usage

callback_set(
  classname,
  on_begin = NULL,
  on_end = NULL,
  on_exit = NULL,
  on_epoch_begin = NULL,
  on_before_valid = NULL,
  on_epoch_end = NULL,
  on_batch_begin = NULL,
  on_batch_end = NULL,
  on_after_backward = NULL,
  on_batch_valid_begin = NULL,
  on_batch_valid_end = NULL,
  on_valid_end = NULL,
  state_dict = NULL,
  load_state_dict = NULL,
  initialize = NULL,
  public = NULL,
  private = NULL,
  active = NULL,
  parent_env = parent.frame(),
  inherit = CallbackSet,
  lock_objects = FALSE
)

Arguments

Value

CallbackSet

See Also

Other Callback: TorchCallback, as_torch_callback(), as_torch_callbacks(), mlr3torch_callbacks, mlr_callback_set, mlr_callback_set.checkpoint, mlr_callback_set.progress, mlr_callback_set.tb, mlr_callback_set.unfreeze, mlr_context_torch, t_clbk(), torch_callback()

Description

The cross_entropy loss function selects the multi-class (nn_cross_entropy_loss) or binary (nn_bce_with_logits_loss) cross entropy loss based on the number of classes. Because of this, there is a slight reparameterization of the loss arguments, see Parameters.

Parameters

• class_weight:: torch_tensor
  The class weights. For multi-class problems, this must be a torch_tensor of length num_classes (and is passed as argument weight to nn_cross_entropy_loss). For binary problems, this must be a scalar (and is passed as argument pos_weight to nn_bce_with_logits_loss).

• ignore_index:: integer(1)
  Index of the class which to ignore and which does not contribute to the gradient. This is only available for multi-class loss.

• reduction :: character(1)
  The reduction to apply. Is either "mean" or "sum" and passed as argument reduction to either loss function. The default is "mean".

Examples

loss = t_loss("cross_entropy")
# multi-class
multi_ce = loss$generate(tsk("iris"))
multi_ce

# binary
binary_ce = loss$generate(tsk("sonar"))
binary_ce

Description

A data descriptor is a rather internal data structure used in the lazy_tensor data type. In essence it is an annotated torch::dataset and a preprocessing graph (consisting mosty of PipeOpModule operators). The additional meta data (e.g. pointer, shapes) allows to preprocess lazy_tensors in an mlr3pipelines::Graph just like any (non-lazy) data types. The preprocessing is applied when materialize() is called on the lazy_tensor.

To create a data descriptor, you can also use the as_data_descriptor() function.

Details

While it would be more natural to define this as an S3 class, we opted for an R6 class to avoid the usual trouble of serializing S3 objects. If each row contained a DataDescriptor as an S3 class, this would copy the object when serializing.

Public fields

Methods

Public methods

• DataDescriptor$new()

• DataDescriptor$print()

• DataDescriptor$clone()

Method new()

Creates a new instance of this R6 class.

Usage

DataDescriptor$new(
  dataset,
  dataset_shapes = NULL,
  graph = NULL,
  input_map = NULL,
  pointer = NULL,
  pointer_shape = NULL,
  pointer_shape_predict = NULL,
  clone_graph = TRUE
)

Arguments

Method print()

Prints the object

Usage

DataDescriptor$print(...)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

DataDescriptor$clone(deep = FALSE)

Arguments

See Also

ModelDescriptor, lazy_tensor

Examples

# Create a dataset
ds = dataset(
  initialize = function() self$x = torch_randn(10, 3, 3),
  .getitem = function(i) list(x = self$x[i, ]),
  .length = function() nrow(self$x)
)()
dd = DataDescriptor$new(ds, list(x = c(NA, 3, 3)))
dd
# is the same as using the converter:
as_data_descriptor(ds, list(x = c(NA, 3, 3)))

Description

Infer the shapes of the output of a function based on the shapes of the input. This is done as follows:

1. All NAs are replaced with values 1, 2, 3.

2. Three tensors are generated for the three shapes of step 1.

3. The function is called on these three tensors and the shapes are calculated.

4. If:

   • the number of dimensions varies, an error is thrown.

   • the number of dimensions is the same, values are set to NA if the dimension is varying between the three tensors and otherwise set to the unique value.

Usage

infer_shapes(shapes_in, param_vals, output_names, fn, rowwise, id)

Arguments

Value

(list())
A list of shapes of the output tensors.

Description

Represents an entry point representing a tensor containing all categorical (factor(), ordered(), logical()) features of a task.

Usage

ingress_categ(shape = NULL)

Arguments

Value

TorchIngressToken

Description

Represents an entry point representing a tensor containing a single lazy tensor feature.

Usage

ingress_ltnsr(feature_name = NULL, shape = NULL)

Arguments

Value

TorchIngressToken

Description

Represents an entry point representing a tensor containing all numeric (integer() and double()) features of a task.

Usage

ingress_num(shape = NULL)

Arguments

Value

TorchIngressToken

Description

Checks whether an object is a lazy tensor.

Usage

is_lazy_tensor(x)

Arguments

Description

Shape of a lazy tensor. Might be NULL if the shapes is not known or varying between rows. Batch dimension is always NA.

Usage

lazy_shape(x)

Arguments

Value

(integer() or NULL)

Examples

lt = as_lazy_tensor(1:10)
lazy_shape(lt)
lt = as_lazy_tensor(matrix(1:10, nrow = 2))
lazy_shape(lt)

Description

Create a lazy tensor.

Usage

lazy_tensor(data_descriptor = NULL, ids = NULL)

Arguments

Examples

ds = dataset("example",
  initialize = function() self$iris = iris[, -5],
  .getitem = function(i) list(x = torch_tensor(as.numeric(self$iris[i, ]))),
  .length = function() nrow(self$iris)
)()
dd = as_data_descriptor(ds, list(x = c(NA, 4L)))
lt = as_lazy_tensor(dd)

Description

This will materialize a lazy_tensor() or a data.frame() / list() containing – among other things – lazy_tensor() columns. I.e. the data described in the underlying DataDescriptors is loaded for the indices in the lazy_tensor(), is preprocessed and then put unto the specified device. Because not all elements in a lazy tensor must have the same shape, a list of tensors is returned by default. If all elements have the same shape, these tensors can also be rbinded into a single tensor (parameter rbind).

Usage

materialize(x, device = "cpu", rbind = FALSE, ...)

## S3 method for class 'list'
materialize(x, device = "cpu", rbind = FALSE, cache = "auto", ...)

Arguments

Details

Materializing a lazy tensor consists of:

1. Loading the data from the internal dataset of the DataDescriptor.

2. Processing these batches in the preprocessing Graphs.

3. Returning the result of the PipeOp pointed to by the DataDescriptor (pointer).

With multiple lazy_tensor columns we can benefit from caching because: a) Output(s) from the dataset might be input to multiple graphs. b) Different lazy tensors might be outputs from the same graph.

For this reason it is possible to provide a cache environment. The hash key for a) is the hash of the indices and the dataset. The hash key for b) is the hash of the indices, dataset and preprocessing graph.

Value

(list() of lazy_tensors or a lazy_tensor)

Examples

lt1 = as_lazy_tensor(torch_randn(10, 3))
materialize(lt1, rbind = TRUE)
materialize(lt1, rbind = FALSE)
lt2 = as_lazy_tensor(torch_randn(10, 4))
d = data.table::data.table(lt1 = lt1, lt2 = lt2)
materialize(d, rbind = TRUE)
materialize(d, rbind = FALSE)

Description

This lazy data backend wraps a constructor that lazily creates another backend, e.g. by downloading (and caching) some data from the internet. This backend should be used, when some metadata of the backend is known in advance and should be accessible before downloading the actual data. When the backend is first constructed, it is verified that the provided metadata was correct, otherwise an informative error message is thrown. After the construction of the lazily constructed backend, calls like ⁠$data()⁠, ⁠$missings()⁠, ⁠$distinct()⁠, or ⁠$hash()⁠ are redirected to it.

Information that is available before the backend is constructed is:

• nrow - The number of rows (set as the length of the rownames).

• ncol - The number of columns (provided via the id column of col_info).

• colnames - The column names.

• rownames - The row names.

• col_info - The column information, which can be obtained via mlr3::col_info().

Beware that accessing the backend's hash also contructs the backend.

Note that while in most cases the data contains lazy_tensor columns, this is not necessary and the naming of this class has nothing to do with the lazy_tensor data type.

Important

When the constructor generates factor() variables it is important that the ordering of the levels in data corresponds to the ordering of the levels in the col_info argument.

Super class

mlr3::DataBackend -> DataBackendLazy

Active bindings

Methods

Public methods

• DataBackendLazy$new()

• DataBackendLazy$data()

• DataBackendLazy$head()

• DataBackendLazy$distinct()

• DataBackendLazy$missings()

• DataBackendLazy$print()

Method new()

Creates a new instance of this R6 class.

Usage

DataBackendLazy$new(constructor, rownames, col_info, primary_key)

Arguments

Method data()

Returns a slice of the data in the specified format. The rows must be addressed as vector of primary key values, columns must be referred to via column names. Queries for rows with no matching row id and queries for columns with no matching column name are silently ignored. Rows are guaranteed to be returned in the same order as rows, columns may be returned in an arbitrary order. Duplicated row ids result in duplicated rows, duplicated column names lead to an exception.

Accessing the data triggers the construction of the backend.

Usage

DataBackendLazy$data(rows, cols)

Arguments

Method head()

Retrieve the first n rows. This triggers the construction of the backend.

Usage

DataBackendLazy$head(n = 6L)

Arguments

Returns

data.table::data.table() of the first n rows.

Method distinct()

Returns a named list of vectors of distinct values for each column specified. If na_rm is TRUE, missing values are removed from the returned vectors of distinct values. Non-existing rows and columns are silently ignored.

This triggers the construction of the backend.

Usage

DataBackendLazy$distinct(rows, cols, na_rm = TRUE)

Arguments

Returns

Named list() of distinct values.

Method missings()

Returns the number of missing values per column in the specified slice of data. Non-existing rows and columns are silently ignored.

This triggers the construction of the backend.

Usage

DataBackendLazy$missings(rows, cols)

Arguments

Returns

Total of missing values per column (named numeric()).

Method print()

Printer.

Usage

DataBackendLazy$print()

Examples

# We first define a backend constructor
constructor = function(backend) {
  cat("Data is constructed!\n")
  DataBackendDataTable$new(
    data.table(x = rnorm(10), y = rnorm(10), row_id = 1:10),
    primary_key = "row_id"
  )
}

# to wrap this backend constructor in a lazy backend, we need to provide the correct metadata for it
column_info = data.table(
  id = c("x", "y", "row_id"),
  type = c("numeric", "numeric", "integer"),
  levels = list(NULL, NULL, NULL)
)
backend_lazy = DataBackendLazy$new(
  constructor = constructor,
  rownames = 1:10,
  col_info = column_info,
  primary_key = "row_id"
)

# Note that the constructor is not called for the calls below
# as they can be read from the metadata
backend_lazy$nrow
backend_lazy$rownames
backend_lazy$ncol
backend_lazy$colnames
col_info(backend_lazy)

# Only now the backend is constructed
backend_lazy$data(1, "x")
# Is the same as:
backend_lazy$backend$data(1, "x")

Description

Base class from which callbacks should inherit (see section Inheriting). A callback set is a collection of functions that are executed at different stages of the training loop. They can be used to gain more control over the training process of a neural network without having to write everything from scratch.

When used a in torch learner, the CallbackSet is wrapped in a TorchCallback. The latters parameter set represents the arguments of the CallbackSet's ⁠$initialize()⁠ method.

Inheriting

For each available stage (see section Stages) a public method ⁠$on_<stage>()⁠ can be defined. The evaluation context (a ContextTorch) can be accessed via self$ctx, which contains the current state of the training loop. This context is assigned at the beginning of the training loop and removed afterwards. Different stages of a callback can communicate with each other by assigning values to ⁠$self⁠.

State: To be able to store information in the ⁠$model⁠ slot of a LearnerTorch, callbacks support a state API. You can overload the ⁠$state_dict()⁠ public method to define what will be stored in ⁠learner$model$callbacks$<id>⁠ after training finishes. This then also requires to implement a ⁠$load_state_dict(state_dict)⁠ method that defines how to load a previously saved callback state into a different callback. Note that the ⁠$state_dict()⁠ should not include the parameter values that were used to initialize the callback.

For creating custom callbacks, the function torch_callback() is recommended, which creates a CallbackSet and then wraps it in a TorchCallback. To create a CallbackSet the convenience function callback_set() can be used. These functions perform checks such as that the stages are not accidentally misspelled.

Stages

• begin :: Run before the training loop begins.

• epoch_begin :: Run he beginning of each epoch.

• batch_begin :: Run before the forward call.

• after_backward :: Run after the backward call.

• batch_end :: Run after the optimizer step.

• batch_valid_begin :: Run before the forward call in the validation loop.

• batch_valid_end :: Run after the forward call in the validation loop.

• valid_end :: Run at the end of validation.

• epoch_end :: Run at the end of each epoch.

• end :: Run after last epoch.

• exit :: Run at last, using on.exit().

Terminate Training

If training is to be stopped, it is possible to set the field ⁠$terminate⁠ of ContextTorch. At the end of every epoch this field is checked and if it is TRUE, training stops. This can for example be used to implement custom early stopping.

Public fields

Active bindings

Methods

Public methods

• CallbackSet$print()

• CallbackSet$state_dict()

• CallbackSet$load_state_dict()

• CallbackSet$clone()

Method print()

Prints the object.

Usage

CallbackSet$print(...)

Arguments

Method state_dict()

Returns information that is kept in the the LearnerTorch's state after training. This information should be loadable into the callback using ⁠$load_state_dict()⁠ to be able to continue training. This returns NULL by default.

Usage

CallbackSet$state_dict()

Method load_state_dict()

Loads the state dict into the callback to continue training.

Usage

CallbackSet$load_state_dict(state_dict)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

CallbackSet$clone(deep = FALSE)

Arguments

See Also

Other Callback: TorchCallback, as_torch_callback(), as_torch_callbacks(), callback_set(), mlr3torch_callbacks, mlr_callback_set.checkpoint, mlr_callback_set.progress, mlr_callback_set.tb, mlr_callback_set.unfreeze, mlr_context_torch, t_clbk(), torch_callback()

Description

Saves the optimizer and network states during training. The final network and optimizer are always stored.

Details

Saving the learner itself in the callback with a trained model is impossible, as the model slot is set after the last callback step is executed.

Super class

mlr3torch::CallbackSet -> CallbackSetCheckpoint

Methods

Public methods

• CallbackSetCheckpoint$new()

• CallbackSetCheckpoint$on_epoch_end()

• CallbackSetCheckpoint$on_batch_end()

• CallbackSetCheckpoint$on_exit()

• CallbackSetCheckpoint$clone()

Method new()

Creates a new instance of this R6 class.

Usage

CallbackSetCheckpoint$new(path, freq, freq_type = "epoch")

Arguments

Method on_epoch_end()

Saves the network and optimizer state dict. Does nothing if freq_type or freq are not met.

Usage

CallbackSetCheckpoint$on_epoch_end()

Method on_batch_end()

Saves the selected objects defined in save. Does nothing if freq_type or freq are not met.

Usage

CallbackSetCheckpoint$on_batch_end()

Method on_exit()

Saves the learner.

Usage

CallbackSetCheckpoint$on_exit()

Method clone()

The objects of this class are cloneable with this method.

Usage

CallbackSetCheckpoint$clone(deep = FALSE)

Arguments

See Also

Other Callback: TorchCallback, as_torch_callback(), as_torch_callbacks(), callback_set(), mlr3torch_callbacks, mlr_callback_set, mlr_callback_set.progress, mlr_callback_set.tb, mlr_callback_set.unfreeze, mlr_context_torch, t_clbk(), torch_callback()

Examples

cb = t_clbk("checkpoint", freq = 1)
task = tsk("iris")

pth = tempfile()
learner = lrn("classif.mlp", epochs = 3, batch_size = 1, callbacks = cb)
learner$param_set$set_values(cb.checkpoint.path = pth)

learner$train(task)

list.files(pth)

Description

Saves the training and validation history during training. The history is saved as a data.table where the validation measures are prefixed with "valid." and the training measures are prefixed with "train.".

Super class

mlr3torch::CallbackSet -> CallbackSetHistory

Methods

Public methods

• CallbackSetHistory$on_begin()

• CallbackSetHistory$state_dict()

• CallbackSetHistory$load_state_dict()

• CallbackSetHistory$on_before_valid()

• CallbackSetHistory$on_epoch_end()

• CallbackSetHistory$clone()

Method on_begin()

Initializes lists where the train and validation metrics are stored.

Usage

CallbackSetHistory$on_begin()

Method state_dict()

Converts the lists to data.tables.

Usage

CallbackSetHistory$state_dict()

Method load_state_dict()

Sets the field ⁠$train⁠ and ⁠$valid⁠ to those contained in the state dict.

Usage

CallbackSetHistory$load_state_dict(state_dict)

Arguments

Method on_before_valid()

Add the latest training scores to the history.

Usage

CallbackSetHistory$on_before_valid()

Method on_epoch_end()

Add the latest validation scores to the history.

Usage

CallbackSetHistory$on_epoch_end()

Method clone()

The objects of this class are cloneable with this method.

Usage

CallbackSetHistory$clone(deep = FALSE)

Arguments

Examples

cb = t_clbk("history")
task = tsk("iris")

learner = lrn("classif.mlp", epochs = 3, batch_size = 1,
  callbacks = t_clbk("history"), validate = 0.3)
learner$param_set$set_values(
  measures_train = msrs(c("classif.acc", "classif.ce")),
  measures_valid = msr("classif.ce")
)
learner$train(task)

print(learner$model$callbacks$history)

Description

Changes the learning rate based on the schedule specified by a torch::lr_scheduler.

As of this writing, the following are available:

• torch::lr_cosine_annealing()

• torch::lr_lambda()

• torch::lr_multiplicative()

• torch::lr_one_cycle() (where the default values for epochs and steps_per_epoch are the number of training epochs and the number of batches per epoch)

• torch::lr_reduce_on_plateau()

• torch::lr_step()

• Custom schedulers defined with torch::lr_scheduler().

Super class

mlr3torch::CallbackSet -> CallbackSetLRScheduler

Public fields

Methods

Public methods

• CallbackSetLRScheduler$new()

• CallbackSetLRScheduler$on_begin()

• CallbackSetLRScheduler$clone()

Method new()

Creates a new instance of this R6 class.

Usage

CallbackSetLRScheduler$new(.scheduler, step_on_epoch, ...)

Arguments

Method on_begin()

Creates the scheduler using the optimizer from the context

Usage

CallbackSetLRScheduler$on_begin()

Method clone()

The objects of this class are cloneable with this method.

Usage

CallbackSetLRScheduler$clone(deep = FALSE)

Arguments

Description

Changes the learning rate based on the 1cycle learning rate policy.

Wraps torch::lr_one_cycle(), where the default values for epochs and steps_per_epoch are the number of training epochs and the number of batches per epoch.

Super classes

mlr3torch::CallbackSet -> mlr3torch::CallbackSetLRScheduler -> CallbackSetLRSchedulerOneCycle

Methods

Public methods

• CallbackSetLRSchedulerOneCycle$new()

• CallbackSetLRSchedulerOneCycle$on_begin()

• CallbackSetLRSchedulerOneCycle$clone()

Method new()

Creates a new instance of this R6 class.

Usage

CallbackSetLRSchedulerOneCycle$new(...)

Arguments

Method on_begin()

Creates the scheduler using the optimizer from the context

Usage

CallbackSetLRSchedulerOneCycle$on_begin()

Method clone()

The objects of this class are cloneable with this method.

Usage

CallbackSetLRSchedulerOneCycle$clone(deep = FALSE)

Arguments

Description

Reduces the learning rate when the first validation metric stops improving for patience epochs. Wraps torch::lr_reduce_on_plateau()

Super classes

mlr3torch::CallbackSet -> mlr3torch::CallbackSetLRScheduler -> CallbackSetLRSchedulerReduceOnPlateau

Methods

Public methods

• CallbackSetLRSchedulerReduceOnPlateau$new()

• CallbackSetLRSchedulerReduceOnPlateau$clone()

Method new()

Creates a new instance of this R6 class.

Usage

CallbackSetLRSchedulerReduceOnPlateau$new(...)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

CallbackSetLRSchedulerReduceOnPlateau$clone(deep = FALSE)

Arguments

Description

Prints a progress bar and the metrics for training and validation.

Super class

mlr3torch::CallbackSet -> CallbackSetProgress

Methods

Public methods

• CallbackSetProgress$new()

• CallbackSetProgress$on_epoch_begin()

• CallbackSetProgress$on_batch_end()

• CallbackSetProgress$on_before_valid()

• CallbackSetProgress$on_batch_valid_end()

• CallbackSetProgress$on_epoch_end()

• CallbackSetProgress$on_end()

• CallbackSetProgress$clone()

Method new()

Creates a new instance of this R6 class.

Usage

CallbackSetProgress$new(digits = 2)

Arguments

Method on_epoch_begin()

Initializes the progress bar for training.

Usage

CallbackSetProgress$on_epoch_begin()

Method on_batch_end()

Increments the training progress bar.

Usage

CallbackSetProgress$on_batch_end()

Method on_before_valid()

Creates the progress bar for validation.

Usage

CallbackSetProgress$on_before_valid()

Method on_batch_valid_end()

Increments the validation progress bar.

Usage

CallbackSetProgress$on_batch_valid_end()

Method on_epoch_end()

Prints a summary of the training and validation process.

Usage

CallbackSetProgress$on_epoch_end()

Method on_end()

Prints the time at the end of training.

Usage

CallbackSetProgress$on_end()

Method clone()

The objects of this class are cloneable with this method.

Usage

CallbackSetProgress$clone(deep = FALSE)

Arguments

See Also

Other Callback: TorchCallback, as_torch_callback(), as_torch_callbacks(), callback_set(), mlr3torch_callbacks, mlr_callback_set, mlr_callback_set.checkpoint, mlr_callback_set.tb, mlr_callback_set.unfreeze, mlr_context_torch, t_clbk(), torch_callback()

Examples

task = tsk("iris")

learner = lrn("classif.mlp", epochs = 5, batch_size = 1,
  callbacks = t_clbk("progress"), validate = 0.3)
learner$param_set$set_values(
  measures_train = msrs(c("classif.acc", "classif.ce")),
  measures_valid = msr("classif.ce")
)

learner$train(task)

Description

Logs training loss, training measures, and validation measures as events. To view them, use TensorBoard with tensorflow::tensorboard() (requires tensorflow) or the CLI.

Details

Logs events at most every epoch.

Super class

mlr3torch::CallbackSet -> CallbackSetTB

Methods

Public methods

• CallbackSetTB$new()

• CallbackSetTB$on_epoch_end()

• CallbackSetTB$clone()

Method new()

Creates a new instance of this R6 class.

Usage

CallbackSetTB$new(path, log_train_loss)

Arguments

Method on_epoch_end()

Logs the training loss, training measures, and validation measures as TensorBoard events.

Usage

CallbackSetTB$on_epoch_end()

Method clone()

The objects of this class are cloneable with this method.

Usage

CallbackSetTB$clone(deep = FALSE)

Arguments

See Also

Other Callback: TorchCallback, as_torch_callback(), as_torch_callbacks(), callback_set(), mlr3torch_callbacks, mlr_callback_set, mlr_callback_set.checkpoint, mlr_callback_set.progress, mlr_callback_set.unfreeze, mlr_context_torch, t_clbk(), torch_callback()

Description

Unfreeze some weights (parameters of the network) after some number of steps or epochs.

Super class

mlr3torch::CallbackSet -> CallbackSetUnfreeze

Methods

Public methods

• CallbackSetUnfreeze$new()

• CallbackSetUnfreeze$on_begin()

• CallbackSetUnfreeze$on_epoch_begin()

• CallbackSetUnfreeze$on_batch_begin()

• CallbackSetUnfreeze$clone()

Method new()

Creates a new instance of this R6 class.

Usage

CallbackSetUnfreeze$new(starting_weights, unfreeze)

Arguments

Method on_begin()

Sets the starting weights

Usage

CallbackSetUnfreeze$on_begin()

Method on_epoch_begin()

Unfreezes weights if the training is at the correct epoch

Usage

CallbackSetUnfreeze$on_epoch_begin()

Method on_batch_begin()

Unfreezes weights if the training is at the correct batch

Usage

CallbackSetUnfreeze$on_batch_begin()

Method clone()

The objects of this class are cloneable with this method.

Usage

CallbackSetUnfreeze$clone(deep = FALSE)

Arguments

See Also

Other Callback: TorchCallback, as_torch_callback(), as_torch_callbacks(), callback_set(), mlr3torch_callbacks, mlr_callback_set, mlr_callback_set.checkpoint, mlr_callback_set.progress, mlr_callback_set.tb, mlr_context_torch, t_clbk(), torch_callback()

Examples

task = tsk("iris")
cb = t_clbk("unfreeze")
mlp = lrn("classif.mlp", callbacks = cb,
 cb.unfreeze.starting_weights = select_invert(
   select_name(c("0.weight", "3.weight", "6.weight", "6.bias"))
 ),
 cb.unfreeze.unfreeze = data.table(
   epoch = c(2, 5),
   weights = list(select_name("0.weight"), select_name(c("3.weight", "6.weight")))
 ),
 epochs = 6, batch_size = 150, neurons = c(1, 1, 1)
)

mlp$train(task)

Description

Context for training a torch learner. This is the - mostly read-only - information callbacks have access to through the argument ctx. For more information on callbacks, see CallbackSet.

Public fields

Methods

Public methods

• ContextTorch$new()

• ContextTorch$clone()

Method new()

Creates a new instance of this R6 class.

Usage

ContextTorch$new(
  learner,
  task_train,
  task_valid = NULL,
  loader_train,
  loader_valid = NULL,
  measures_train = NULL,
  measures_valid = NULL,
  network,
  optimizer,
  loss_fn,
  total_epochs,
  prediction_encoder,
  eval_freq = 1L,
  device
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

ContextTorch$clone(deep = FALSE)

Arguments

See Also

Other Callback: TorchCallback, as_torch_callback(), as_torch_callbacks(), callback_set(), mlr3torch_callbacks, mlr_callback_set, mlr_callback_set.checkpoint, mlr_callback_set.progress, mlr_callback_set.tb, mlr_callback_set.unfreeze, t_clbk(), torch_callback()

Description

This base class provides the basic functionality for training and prediction of a neural network. All torch learners should inherit from this class.

Validation

To specify the validation data, you can set the ⁠$validate⁠ field of the Learner, which can be set to:

• NULL: no validation

• ratio: only proportion 1 - ratio of the task is used for training and ratio is used for validation.

• "test" means that the "test" task of a resampling is used and is not possible when calling ⁠$train()⁠ manually.

• "predefined": This will use the predefined ⁠$internal_valid_task⁠ of a mlr3::Task.

This validation data can also be used for early stopping, see the description of the Learner's parameters.

Saving a Learner

In order to save a LearnerTorch for later usage, it is necessary to call the ⁠$marshal()⁠ method on the Learner before writing it to disk, as the object will otherwise not be saved correctly. After loading a marshaled LearnerTorch into R again, you then need to call ⁠$unmarshal()⁠ to transform it into a useable state.

Early Stopping and Internal Tuning

In order to prevent overfitting, the LearnerTorch class allows to use early stopping via the patience and min_delta parameters, see the Learner's parameters. When tuning a LearnerTorch it is also possible to combine the explicit tuning via mlr3tuning and the LearnerTorch's internal tuning of the epochs via early stopping. To do so, you just need to include ⁠epochs = to_tune(upper = <upper>, internal = TRUE)⁠ in the search space, where ⁠<upper>⁠ is the maximally allowed number of epochs, and configure the early stopping.

Network Head and Target Encoding

Torch learners are expected to have the following output:

• binary classification: ⁠(batch_size, 1)⁠, representing the logits for the positive class.

• multiclass classification: ⁠(batch_size, n_classes)⁠, representing the logits for all classes.

• regression: ⁠(batch_size, 1)⁠ representing the response prediction.

Furthermore, the target encoding is expected to be as follows:

• regression: The numeric target variable of a TaskRegr is encoded as a torch_float with shape c(batch_size, 1).

• binary classification: The factor target variable of a TaskClassif is encoded as a torch_float with shape ⁠(batch_size, 1)⁠ where the positive class (Task$positive, which is also ensured to be the first factor level) is 1 and the negative class is 0.

• multi-class classification: The factor target variable of a TaskClassif is a label-encoded torch_long with shape (batch_size) where the label-encoding goes from 1 to n_classes.

Important Runtime Considerations

There are a few hyperparameters settings that can have a considerable impact on the runtime of the learner. These include:

• device: Use a GPU if possible.

• num_threads: Set this to the number of CPU cores available if training on CPU.

• tensor_dataset: Set this to TRUE (or "device" if on a GPU) if the dataset fits into memory.

• batch_size: Especially for very small models, choose a larger batch size.

Also, see the Early Stopping and Internal Tuning section for how to terminate training early.

Model

The Model is a list of class "learner_torch_model" with the following elements:

• network :: The trained network.

• optimizer :: The ⁠$state_dict()⁠ optimizer used to train the network.

• loss_fn :: The ⁠$state_dict()⁠ of the loss used to train the network.

• callbacks :: The callbacks used to train the network.

• seed :: The seed that was / is used for training and prediction.

• epochs :: How many epochs the model was trained for (early stopping).

• task_col_info :: A data.table() containing information about the train-task.

Parameters

General:

The parameters of the optimizer, loss and callbacks, prefixed with "opt.", "loss." and "cb.<callback id>." respectively, as well as:

• epochs :: integer(1)
  The number of epochs.

• device :: character(1)
  The device. One of "auto", "cpu", or "cuda" or other values defined in mlr_reflections$torch$devices. The value is initialized to "auto", which will select "cuda" if possible, then try "mps" and otherwise fall back to "cpu".

• num_threads :: integer(1)
  The number of threads for intraop pararallelization (if device is "cpu"). This value is initialized to 1.

• num_interop_threads :: integer(1)
  The number of threads for intraop and interop pararallelization (if device is "cpu"). This value is initialized to 1. Note that this can only be set once during a session and changing the value within an R session will raise a warning.

• seed :: integer(1) or "random" or NULL
  The torch seed that is used during training and prediction. This value is initialized to "random", which means that a random seed will be sampled at the beginning of the training phase. This seed (either set or randomly sampled) is available via ⁠$model$seed⁠ after training and used during prediction. Note that by setting the seed during the training phase this will mean that by default (i.e. when seed is "random"), clones of the learner will use a different seed. If set to NULL, no seeding will be done.

• tensor_dataset :: logical(1) | "device"
  Whether to load all batches at once at the beginning of training and stack them. This is initialized to FALSE. If set to "device", the device of the tensors will be set to the value of device, which can avoid unnecessary moving of tensors between devices. When your dataset fits into memory this will make the loading of batches faster. Note that this should not be set for datasets that contain lazy_tensors with random data augmentation, as this augmentation will only be applied once at the beginning of training.

Evaluation:

• measures_train :: Measure or list() of Measures
  Measures to be evaluated during training.

• measures_valid :: Measure or list() of Measures
  Measures to be evaluated during validation.

• eval_freq :: integer(1)
  How often the train / validation predictions are evaluated using measures_train / measures_valid. This is initialized to 1. Note that the final model is always evaluated.

Early Stopping:

• patience :: integer(1)
  This activates early stopping using the validation scores. If the performance of a model does not improve for patience evaluation steps, training is ended. Note that the final model is stored in the learner, not the best model. This is initialized to 0, which means no early stopping. The first entry from measures_valid is used as the metric. This also requires to specify the ⁠$validate⁠ field of the Learner, as well as measures_valid. If this is set, the epoch after which no improvement was observed, can be accessed via the ⁠$internal_tuned_values⁠ field of the learner.

• min_delta :: double(1)
  The minimum improvement threshold for early stopping. Is initialized to 0.

Dataloader:

• batch_size :: integer(1)
  The batch size (required).

• shuffle :: logical(1)
  Whether to shuffle the instances in the dataset. This is initialized to TRUE, which differs from the default (FALSE).

• sampler :: torch::sampler
  Object that defines how the dataloader draw samples.

• batch_sampler :: torch::sampler
  Object that defines how the dataloader draws batches.

• num_workers :: integer(1)
  The number of workers for data loading (batches are loaded in parallel). The default is 0, which means that data will be loaded in the main process.

• collate_fn :: function
  How to merge a list of samples to form a batch.

• pin_memory :: logical(1)
  Whether the dataloader copies tensors into CUDA pinned memory before returning them.

• drop_last :: logical(1)
  Whether to drop the last training batch in each epoch during training. Default is FALSE.

• timeout :: numeric(1)
  The timeout value for collecting a batch from workers. Negative values mean no timeout and the default is -1.

• worker_init_fn :: ⁠function(id)⁠
  A function that receives the worker id (in ⁠[1, num_workers]⁠) and is exectued after seeding on the worker but before data loading.

• worker_globals :: list() | character()
  When loading data in parallel, this allows to export globals to the workers. If this is a character vector, the objects in the global environment with those names are copied to the workers.

• worker_packages :: character()
  Which packages to load on the workers.

Also see torch::dataloder for more information.

Inheriting

There are no seperate classes for classification and regression to inherit from. Instead, the task_type must be specified as a construction argument. Currently, only classification and regression are supported.

When inheriting from this class, one should overload the following methods:

• .network(task, param_vals)
  (Task, list()) -> nn_module
  Construct a torch::nn_module object for the given task and parameter values, i.e. the neural network that is trained by the learner. Note that a specific output shape is expected from the returned network, see section Network Head and Target Encoding. You can use output_dim_for() to obtain the correct output dimension for a given task.

• .ingress_tokens(task, param_vals)
  (Task, list()) -> named list() with TorchIngressTokens
  Create the TorchIngressTokens that are passed to the task_dataset constructor. The number of ingress tokens must correspond to the number of input parameters of the network. If there is more than one input, the names must correspond to the inputs of the network. See ingress_num, ingress_categ, and ingress_ltnsr on how to easily create the correct tokens. For more flexibility, you can also directly implement the .dataset(task, param_vals) method, see below.

• .dataset(task, param_vals)
  (Task, list()) -> torch::dataset
  Create the dataset for the task. Don't implement this if the .ingress_tokens() method is defined. The dataset must return a named list where:

  • x is a list of torch tensors that are the input to the network. For networks with more than one input, the names must correspond to the inputs of the network.

  • y is the target tensor.

  • .index are the indices of the batch (integer() or a torch_int()).

  For information on the expected target encoding of y, see section Network Head and Target Encoding. Moreover, one needs to pay attention respect the row ids of the provided task. It is recommended to relu on task_dataset for creating the dataset.

It is also possible to overwrite the private .dataloader() method. This must respect the dataloader parameters from the ParamSet.

• .dataloader(dataset, param_vals)
  (Task, list()) -> torch::dataloader
  Create a dataloader from the task. Needs to respect at least batch_size and shuffle (otherwise predictions will be incorrectly ordered).

To change the predict types, it is possible to overwrite the method below:

• .encode_prediction(predict_tensor, task)
  (torch_tensor, Task) -> list()
  Take in the raw predictions from self$network (predict_tensor) and encode them into a format that can be converted to valid mlr3 predictions using mlr3::as_prediction_data(). This method must take self$predict_type into account.

While it is possible to add parameters by specifying the param_set construction argument, it is currently not possible to remove existing parameters, i.e. those listed in section Parameters. None of the parameters provided in param_set can have an id that starts with "loss.", ⁠"opt.", or ⁠"cb."', as these are preserved for the dynamically constructed parameters of the optimizer, the loss function, and the callbacks.

To perform additional input checks on the task, the private .check_train_task(task, param_vals) and .check_predict_task(task, param_vals) can be overwritten. These should return TRUE if the input task is valid and otherwise a string with an error message.

For learners that have other construction arguments that should change the hash of a learner, it is required to implement the private ⁠$.additional_phash_input()⁠.

Super class

mlr3::Learner -> LearnerTorch

Active bindings

Methods

Public methods

• LearnerTorch$new()

• LearnerTorch$format()

• LearnerTorch$print()

• LearnerTorch$marshal()

• LearnerTorch$unmarshal()

• LearnerTorch$dataset()

• LearnerTorch$clone()

Method new()

Creates a new instance of this R6 class.

Usage

LearnerTorch$new(
  id,
  task_type,
  param_set,
  properties = character(),
  man,
  label,
  feature_types,
  optimizer = NULL,
  loss = NULL,
  packages = character(),
  predict_types = NULL,
  callbacks = list(),
  jittable = FALSE
)

Arguments

Method format()

Helper for print outputs.

Usage

LearnerTorch$format(...)

Arguments

Method print()

Prints the object.

Usage

LearnerTorch$print(...)

Arguments

Method marshal()

Marshal the learner.

Usage

LearnerTorch$marshal(...)

Arguments

Returns

self

Method unmarshal()

Unmarshal the learner.

Usage

LearnerTorch$unmarshal(...)

Arguments

Returns

self

Method dataset()

Create the dataset for a task.

Usage

LearnerTorch$dataset(task)

Arguments

Returns

dataset

Method clone()

The objects of this class are cloneable with this method.

Usage

LearnerTorch$clone(deep = FALSE)

Arguments

See Also

Other Learner: mlr_learners.ft_transformer, mlr_learners.mlp, mlr_learners.module, mlr_learners.tab_resnet, mlr_learners.torch_featureless, mlr_learners_torch_image, mlr_learners_torch_model

Description

Base Class for Image Learners. The features are assumed to be a single lazy_tensor column in RGB format.

Parameters

Parameters include those inherited from LearnerTorch and the param_set construction argument.

Super classes

mlr3::Learner -> mlr3torch::LearnerTorch -> LearnerTorchImage

Methods

Public methods

• LearnerTorchImage$new()

• LearnerTorchImage$clone()

Method new()

Creates a new instance of this R6 class.

Usage

LearnerTorchImage$new(
  id,
  task_type,
  param_set = ps(),
  label,
  optimizer = NULL,
  loss = NULL,
  callbacks = list(),
  packages,
  man,
  properties = NULL,
  predict_types = NULL,
  jittable = FALSE
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

LearnerTorchImage$clone(deep = FALSE)

Arguments

See Also

Other Learner: mlr_learners.ft_transformer, mlr_learners.mlp, mlr_learners.module, mlr_learners.tab_resnet, mlr_learners.torch_featureless, mlr_learners_torch, mlr_learners_torch_model

Description

Create a torch learner from an instantiated nn_module(). For classification, the output of the network must be the scores (before the softmax).

Parameters

See LearnerTorch

Super classes

mlr3::Learner -> mlr3torch::LearnerTorch -> LearnerTorchModel

Active bindings

Methods

Public methods

• LearnerTorchModel$new()

• LearnerTorchModel$clone()

Method new()

Creates a new instance of this R6 class.

Usage

LearnerTorchModel$new(
  network = NULL,
  ingress_tokens = NULL,
  task_type,
  properties = NULL,
  optimizer = NULL,
  loss = NULL,
  callbacks = list(),
  packages = character(0),
  feature_types = NULL
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

LearnerTorchModel$clone(deep = FALSE)

Arguments

See Also

Other Learner: mlr_learners.ft_transformer, mlr_learners.mlp, mlr_learners.module, mlr_learners.tab_resnet, mlr_learners.torch_featureless, mlr_learners_torch, mlr_learners_torch_image

Other Graph Network: ModelDescriptor(), TorchIngressToken(), mlr_pipeops_module, mlr_pipeops_torch, mlr_pipeops_torch_ingress, mlr_pipeops_torch_ingress_categ, mlr_pipeops_torch_ingress_ltnsr, mlr_pipeops_torch_ingress_num, model_descriptor_to_learner(), model_descriptor_to_module(), model_descriptor_union(), nn_graph()

Examples

# We show the learner using a classification task

# The iris task has 4 features and 3 classes
network = nn_linear(4, 3)
task = tsk("iris")

# This defines the dataloader.
# It loads all 4 features, which are also numeric.
# The shape is (NA, 4) because the batch dimension is generally NA
ingress_tokens = list(
  input = TorchIngressToken(task$feature_names, batchgetter_num, c(NA, 4))
)

# Creating the learner and setting required parameters
learner = lrn("classif.torch_model",
  network = network,
  ingress_tokens = ingress_tokens,
  batch_size = 16,
  epochs = 1,
  device = "cpu"
)

# A simple train-predict
ids = partition(task)
learner$train(task, ids$train)
learner$predict(task, ids$test)

Description

Feature-Tokenizer Transformer for tabular data that can either work on lazy_tensor inputs or on standard tabular features.

Some differences from the paper implementation: no attention compression, no option to have prenormalization in the first layer.

If training is unstable, consider a combination of standardizing features (e.g. using po("scale")), using an adaptive optimizer (e.g. Adam), reducing the learning rate, and using a learning rate scheduler (see CallbackSetLRScheduler for options).

Dictionary

This Learner can be instantiated using the sugar function lrn():

lrn("classif.ft_transformer", ...)
lrn("regr.ft_transformer", ...)

Properties

• Supported task types: 'classif', 'regr'

• Predict Types:

  • classif: 'response', 'prob'

  • regr: 'response'

• Feature Types: “logical”, “integer”, “numeric”, “factor”, “ordered”, “lazy_tensor”

• Required Packages: mlr3, mlr3torch, torch

Parameters

Parameters from LearnerTorch and PipeOpTorchFTTransformerBlock, as well as:

• n_blocks :: integer(1)
  The number of transformer blocks.

• d_token :: integer(1)
  The dimension of the embedding.

• cardinalities :: integer(1)
  The number of categories for each categorical feature. This only needs to be specified when working with lazy_tensor inputs.

• init_token :: character(1)
  The initialization method for the embedding weights. Either "uniform" or "normal". "Uniform" by default.

• ingress_tokens :: named list() or NULL
  A list of TorchIngressTokens. Only required when using lazy tensor features. The names are either "num.input" or "categ.input", and the values are lazy tensor ingress tokens constructed by, e.g. ⁠ingress_ltnsr(<num_feat_name>)⁠.

Super classes

mlr3::Learner -> mlr3torch::LearnerTorch -> LearnerTorchFTTransformer

Methods

Public methods

• LearnerTorchFTTransformer$new()

• LearnerTorchFTTransformer$clone()

Method new()

Creates a new instance of this R6 class.

Usage

LearnerTorchFTTransformer$new(
  task_type,
  optimizer = NULL,
  loss = NULL,
  callbacks = list()
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

LearnerTorchFTTransformer$clone(deep = FALSE)

Arguments

References

Gorishniy Y, Rubachev I, Khrulkov V, Babenko A (2021). “Revisiting Deep Learning for Tabular Data.” arXiv, 2106.11959.

See Also

Other Learner: mlr_learners.mlp, mlr_learners.module, mlr_learners.tab_resnet, mlr_learners.torch_featureless, mlr_learners_torch, mlr_learners_torch_image, mlr_learners_torch_model

Examples

# Define the Learner and set parameter values
learner = lrn("classif.ft_transformer")
learner$param_set$set_values(
  epochs = 1, batch_size = 16, device = "cpu",
  n_blocks = 2, d_token = 32, ffn_d_hidden_multiplier = 4/3
)

# Define a Task
task = tsk("iris")

# Create train and test set
ids = partition(task)

# Train the learner on the training ids
learner$train(task, row_ids = ids$train)

# Make predictions for the test rows
predictions = learner$predict(task, row_ids = ids$test)

# Score the predictions
predictions$score()

Description

Fully connected feed forward network with dropout after each activation function. The features can either be a single lazy_tensor or one or more numeric columns (but not both).

Dictionary

This Learner can be instantiated using the sugar function lrn():

lrn("classif.mlp", ...)
lrn("regr.mlp", ...)

Properties

• Supported task types: 'classif', 'regr'

• Predict Types:

  • classif: 'response', 'prob'

  • regr: 'response'

• Feature Types: “integer”, “numeric”, “lazy_tensor”

• Required Packages: mlr3, mlr3torch, torch

Parameters

Parameters from LearnerTorch, as well as:

• activation :: ⁠[nn_module]⁠
  The activation function. Is initialized to nn_relu.

• activation_args :: named list()
  A named list with initialization arguments for the activation function. This is intialized to an empty list.

• neurons :: integer()
  The number of neurons per hidden layer. By default there is no hidden layer. Setting this to c(10, 20) would have a the first hidden layer with 10 neurons and the second with 20.

• n_layers :: integer()
  The number of layers. This parameter must only be set when neurons has length 1.

• p :: numeric(1)
  The dropout probability. Is initialized to 0.5.

• shape :: integer() or NULL
  The input shape of length 2, e.g. c(NA, 5). Only needs to be present when there is a lazy tensor input with unknown shape (NULL). Otherwise the input shape is inferred from the number of numeric features.

Super classes

mlr3::Learner -> mlr3torch::LearnerTorch -> LearnerTorchMLP

Methods

Public methods

• LearnerTorchMLP$new()

• LearnerTorchMLP$clone()

Method new()

Creates a new instance of this R6 class.

Usage

LearnerTorchMLP$new(
  task_type,
  optimizer = NULL,
  loss = NULL,
  callbacks = list()
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

LearnerTorchMLP$clone(deep = FALSE)

Arguments

References

Gorishniy Y, Rubachev I, Khrulkov V, Babenko A (2021). “Revisiting Deep Learning for Tabular Data.” arXiv, 2106.11959.

See Also

Other Learner: mlr_learners.ft_transformer, mlr_learners.module, mlr_learners.tab_resnet, mlr_learners.torch_featureless, mlr_learners_torch, mlr_learners_torch_image, mlr_learners_torch_model

Examples

# Define the Learner and set parameter values
learner = lrn("classif.mlp")
learner$param_set$set_values(
  epochs = 1, batch_size = 16, device = "cpu",
  neurons = 10
)

# Define a Task
task = tsk("iris")

# Create train and test set
ids = partition(task)

# Train the learner on the training ids
learner$train(task, row_ids = ids$train)

# Make predictions for the test rows
predictions = learner$predict(task, row_ids = ids$test)

# Score the predictions
predictions$score()

Description

Create a torch learner from a torch module.

Dictionary

This Learner can be instantiated using the sugar function lrn():

lrn("classif.module", ...)
lrn("regr.module", ...)

Properties

• Supported task types: 'classif', 'regr'

• Predict Types:

  • classif: 'response', 'prob'

  • regr: 'response'

• Feature Types: “logical”, “integer”, “numeric”, “character”, “factor”, “ordered”, “POSIXct”, “Date”, “lazy_tensor”

• Required Packages: mlr3, mlr3torch, torch

Super classes

mlr3::Learner -> mlr3torch::LearnerTorch -> LearnerTorchModule

Methods

Public methods

• LearnerTorchModule$new()

• LearnerTorchModule$clone()

Method new()

Creates a new instance of this R6 class.

Usage

LearnerTorchModule$new(
  module_generator = NULL,
  param_set = NULL,
  ingress_tokens = NULL,
  task_type,
  properties = NULL,
  optimizer = NULL,
  loss = NULL,
  callbacks = list(),
  packages = character(0),
  feature_types = NULL,
  predict_types = NULL
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

LearnerTorchModule$clone(deep = FALSE)

Arguments

See Also

Other Learner: mlr_learners.ft_transformer, mlr_learners.mlp, mlr_learners.tab_resnet, mlr_learners.torch_featureless, mlr_learners_torch, mlr_learners_torch_image, mlr_learners_torch_model

Other Learner: mlr_learners.ft_transformer, mlr_learners.mlp, mlr_learners.tab_resnet, mlr_learners.torch_featureless, mlr_learners_torch, mlr_learners_torch_image, mlr_learners_torch_model

Examples

nn_one_layer = nn_module("nn_one_layer",
  initialize = function(task, size_hidden) {
    self$first = nn_linear(task$n_features, size_hidden)
    self$second = nn_linear(size_hidden, output_dim_for(task))
  },
  # argument x corresponds to the ingress token x
  forward = function(x) {
    x = self$first(x)
    x = nnf_relu(x)
    self$second(x)
  }
)
learner = lrn("classif.module",
  module_generator = nn_one_layer,
  ingress_tokens = list(x = ingress_num()),
  epochs = 10,
  size_hidden = 20,
  batch_size = 16
)
task = tsk("iris")
learner$train(task)
learner$network

Description

Tabular resnet.

Dictionary

This Learner can be instantiated using the sugar function lrn():

lrn("classif.tab_resnet", ...)
lrn("regr.tab_resnet", ...)

Properties

• Supported task types: 'classif', 'regr'

• Predict Types:

  • classif: 'response', 'prob'

  • regr: 'response'

• Feature Types: “integer”, “numeric”, “lazy_tensor”

• Required Packages: mlr3, mlr3torch, torch

Parameters

Parameters from LearnerTorch, as well as:

• n_blocks :: integer(1)
  The number of blocks.

• d_block :: integer(1)
  The input and output dimension of a block.

• d_hidden :: integer(1)
  The latent dimension of a block.

• d_hidden_multiplier :: numeric(1)
  Alternative way to specify the latent dimension as d_block * d_hidden_multiplier.

• dropout1 :: numeric(1)
  First dropout ratio.

• dropout2 :: numeric(1)
  Second dropout ratio.

• shape :: integer() or NULL
  Shape of the input tensor. Only needs to be provided if the input is a lazy tensor with unknown shape.

Super classes

mlr3::Learner -> mlr3torch::LearnerTorch -> LearnerTorchTabResNet

Methods

Public methods

• LearnerTorchTabResNet$new()

• LearnerTorchTabResNet$clone()

Method new()

Creates a new instance of this R6 class.

Usage

LearnerTorchTabResNet$new(
  task_type,
  optimizer = NULL,
  loss = NULL,
  callbacks = list()
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

LearnerTorchTabResNet$clone(deep = FALSE)

Arguments

References

Gorishniy Y, Rubachev I, Khrulkov V, Babenko A (2021). “Revisiting Deep Learning for Tabular Data.” arXiv, 2106.11959.

See Also

Other Learner: mlr_learners.ft_transformer, mlr_learners.mlp, mlr_learners.module, mlr_learners.torch_featureless, mlr_learners_torch, mlr_learners_torch_image, mlr_learners_torch_model

Examples

# Define the Learner and set parameter values
learner = lrn("classif.tab_resnet")
learner$param_set$set_values(
  epochs = 1, batch_size = 16, device = "cpu",
  n_blocks = 2, d_block = 10, d_hidden = 20, dropout1 = 0.3, dropout2 = 0.3
)

# Define a Task
task = tsk("iris")

# Create train and test set
ids = partition(task)

# Train the learner on the training ids
learner$train(task, row_ids = ids$train)

# Make predictions for the test rows
predictions = learner$predict(task, row_ids = ids$test)

# Score the predictions
predictions$score()

Description

Featureless torch learner. Output is a constant weight that is learned during training. For classification, this should (asymptoptically) result in a majority class prediction when using the standard cross-entropy loss. For regression, this should result in the median for L1 loss and in the mean for L2 loss.

Dictionary

This Learner can be instantiated using the sugar function lrn():

lrn("classif.torch_featureless", ...)
lrn("regr.torch_featureless", ...)

Properties

• Supported task types: 'classif', 'regr'

• Predict Types:

  • classif: 'response', 'prob'

  • regr: 'response'

• Feature Types: “logical”, “integer”, “numeric”, “character”, “factor”, “ordered”, “POSIXct”, “Date”, “lazy_tensor”

• Required Packages: mlr3, mlr3torch, torch

Parameters

Only those from LearnerTorch.

Super classes

mlr3::Learner -> mlr3torch::LearnerTorch -> LearnerTorchFeatureless

Methods

Public methods

• LearnerTorchFeatureless$new()

• LearnerTorchFeatureless$clone()

Method new()

Creates a new instance of this R6 class.

Usage

LearnerTorchFeatureless$new(
  task_type,
  optimizer = NULL,
  loss = NULL,
  callbacks = list()
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

LearnerTorchFeatureless$clone(deep = FALSE)

Arguments

See Also

Other Learner: mlr_learners.ft_transformer, mlr_learners.mlp, mlr_learners.module, mlr_learners.tab_resnet, mlr_learners_torch, mlr_learners_torch_image, mlr_learners_torch_model

Examples

# Define the Learner and set parameter values
learner = lrn("classif.torch_featureless")
learner$param_set$set_values(
  epochs = 1, batch_size = 16, device = "cpu"
  
)

# Define a Task
task = tsk("iris")

# Create train and test set
ids = partition(task)

# Train the learner on the training ids
learner$train(task, row_ids = ids$train)

# Make predictions for the test rows
predictions = learner$predict(task, row_ids = ids$test)

# Score the predictions
predictions$score()

Description

Classic image classification networks from torchvision.

Parameters

Parameters from LearnerTorchImage and

• pretrained :: logical(1)
  Whether to use the pretrained model. The final linear layer will be replaced with a new nn_linear with the number of classes inferred from the Task.

Properties

• Supported task types: "classif"

• Predict Types: "response" and "prob"

• Feature Types: "lazy_tensor"

• Required packages: "mlr3torch", "torch", "torchvision"

Super classes

mlr3::Learner -> mlr3torch::LearnerTorch -> mlr3torch::LearnerTorchImage -> LearnerTorchVision

Methods

Public methods

• LearnerTorchVision$new()

• LearnerTorchVision$clone()

Method new()

Creates a new instance of this R6 class.

Usage

LearnerTorchVision$new(
  name,
  module_generator,
  label,
  optimizer = NULL,
  loss = NULL,
  callbacks = list(),
  jittable = FALSE
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

LearnerTorchVision$clone(deep = FALSE)

Arguments

References

Krizhevsky, Alex, Sutskever, Ilya, Hinton, E. G (2017). “Imagenet classification with deep convolutional neural networks.” Communications of the ACM, 60(6), 84–90. Sandler, Mark, Howard, Andrew, Zhu, Menglong, Zhmoginov, Andrey, Chen, Liang-Chieh (2018). “Mobilenetv2: Inverted residuals and linear bottlenecks.” In Proceedings of the IEEE conference on computer vision and pattern recognition, 4510–4520. He, Kaiming, Zhang, Xiangyu, Ren, Shaoqing, Sun, Jian (2016). “Deep residual learning for image recognition.” In Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778. Simonyan, Karen, Zisserman, Andrew (2014). “Very deep convolutional networks for large-scale image recognition.” arXiv preprint arXiv:1409.1556.

Description

Calls torchvision::transform_center_crop, see there for more information on the parameters. The preprocessing is applied to each element of a batch individually.

Format

R6Class inheriting from PipeOpTaskPreprocTorch.

Construction

po("augment_center_crop"")

Parameters

Description

Calls torchvision::transform_color_jitter, see there for more information on the parameters. The preprocessing is applied to each element of a batch individually.

Format

R6Class inheriting from PipeOpTaskPreprocTorch.

Construction

po("augment_color_jitter"")

Parameters

Description

Calls torchvision::transform_crop, see there for more information on the parameters. The preprocessing is applied to each element of a batch individually.

Format

R6Class inheriting from PipeOpTaskPreprocTorch.

Construction

po("augment_crop"")

Parameters

Description

Calls torchvision::transform_hflip, see there for more information on the parameters. The preprocessing is applied to each element of a batch individually.

Format

R6Class inheriting from PipeOpTaskPreprocTorch.

Construction

po("augment_hflip"")

Parameters

Description

Calls torchvision::transform_random_affine, see there for more information on the parameters. The preprocessing is applied to each element of a batch individually.

Format

R6Class inheriting from PipeOpTaskPreprocTorch.

Construction

po("augment_random_affine"")

Parameters

Description

Calls torchvision::transform_random_choice, see there for more information on the parameters. The preprocessing is applied to each element of a batch individually.

Format

R6Class inheriting from PipeOpTaskPreprocTorch.

Construction

po("augment_random_choice"")

Parameters