Package 'mlr3torch'

Description

Deep Learning library that extends the mlr3 framework by building upon the 'torch' package. It allows to conveniently build, train, and evaluate deep learning models without having to worry about low level details. Custom architectures can be created using the graph language defined in 'mlr3pipelines'.

Options

• mlr3torch.cache: Whether to cache the downloaded data (TRUE) or not (FALSE, default). This can also be set to a specific folder on the file system to be used as the cache directory.

Author(s)

Maintainer: Sebastian Fischer [email protected] (ORCID)

Authors:

• Martin Binder [email protected]

Other contributors:

• Bernd Bischl [email protected] (ORCID) [contributor]

• Lukas Burk [email protected] (ORCID) [contributor]

• Florian Pfisterer [email protected] (ORCID) [contributor]

See Also

Useful links:

• https://mlr3torch.mlr-org.com/

• https://github.com/mlr-org/mlr3torch/

• Report bugs at https://github.com/mlr-org/mlr3torch/issues

Description

Converts the input to a DataDescriptor.

Usage

as_data_descriptor(x, dataset_shapes, ...)

Arguments

Examples

ds = dataset("example",
  initialize = function() self$iris = iris[, -5],
  .getitem = function(i) list(x = torch_tensor(as.numeric(self$iris[i, ]))),
  .length = function() nrow(self$iris)
)()
as_data_descriptor(ds, list(x = c(NA, 4L)))

# if the dataset has a .getbatch method, the shapes are inferred
ds2 = dataset("example",
  initialize = function() self$iris = iris[, -5],
  .getbatch = function(i) list(x = torch_tensor(as.matrix(self$iris[i, ]))),
  .length = function() nrow(self$iris)
)()
as_data_descriptor(ds2)

Description

Convert a object to a lazy_tensor.

Usage

as_lazy_tensor(x, ...)

## S3 method for class 'dataset'
as_lazy_tensor(x, dataset_shapes = NULL, ids = NULL, ...)

Arguments

Examples

iris_ds = dataset("iris",
  initialize = function() {
    self$iris = iris[, -5]
  },
  .getbatch = function(i) {
    list(x = torch_tensor(as.matrix(self$iris[i, ])))
  },
  .length = function() nrow(self$iris)
)()
# no need to specify the dataset shapes as they can be inferred from the .getbatch method
# only first 5 observations
as_lazy_tensor(iris_ds, ids = 1:5)
# all observations
head(as_lazy_tensor(iris_ds))

iris_ds2 = dataset("iris",
  initialize = function() self$iris = iris[, -5],
  .getitem = function(i) list(x = torch_tensor(as.numeric(self$iris[i, ]))),
  .length = function() nrow(self$iris)
)()
# if .getitem is implemented we cannot infer the shapes as they might vary,
# so we have to annotate them explicitly
as_lazy_tensor(iris_ds2, dataset_shapes = list(x = c(NA, 4L)))[1:5]

# Convert a matrix
lt = as_lazy_tensor(matrix(rnorm(100), nrow = 20))
materialize(lt[1:5], rbind = TRUE)

Description

Converts an object to a TorchCallback.

Usage

as_torch_callback(x, clone = FALSE, ...)

Arguments

Value

TorchCallback.

See Also

Other Callback: TorchCallback, as_torch_callbacks(), callback_set(), mlr3torch_callbacks, mlr_callback_set, mlr_callback_set.checkpoint, mlr_callback_set.progress, mlr_context_torch, t_clbk(), torch_callback()

Description

Converts an object to a list of TorchCallback.

Usage

as_torch_callbacks(x, clone, ...)

Arguments

Value

list() of TorchCallbacks

See Also

Other Callback: TorchCallback, as_torch_callback(), callback_set(), mlr3torch_callbacks, mlr_callback_set, mlr_callback_set.checkpoint, mlr_callback_set.progress, mlr_context_torch, t_clbk(), torch_callback()

Other Torch Descriptor: TorchCallback, TorchDescriptor, TorchLoss, TorchOptimizer, as_torch_loss(), as_torch_optimizer(), mlr3torch_losses, mlr3torch_optimizers, t_clbk(), t_loss(), t_opt()

Description

Converts an object to a TorchLoss.

Usage

as_torch_loss(x, clone = FALSE, ...)

Arguments

Value

TorchLoss.

See Also

Other Torch Descriptor: TorchCallback, TorchDescriptor, TorchLoss, TorchOptimizer, as_torch_callbacks(), as_torch_optimizer(), mlr3torch_losses, mlr3torch_optimizers, t_clbk(), t_loss(), t_opt()

Description

Converts an object to a TorchOptimizer.

Usage

as_torch_optimizer(x, clone = FALSE, ...)

Arguments

Value

TorchOptimizer

See Also

Other Torch Descriptor: TorchCallback, TorchDescriptor, TorchLoss, TorchOptimizer, as_torch_callbacks(), as_torch_loss(), mlr3torch_losses, mlr3torch_optimizers, t_clbk(), t_loss(), t_opt()

Description

Asserts whether something is a lazy tensor.

Usage

assert_lazy_tensor(x)

Arguments

Description

First tries cuda, then cpu.

Usage

auto_device(device = NULL)

Arguments

Description

Converts a data frame of categorical data into a long tensor by converting the data to integers. No input checks are performed.

Usage

batchgetter_categ(data, device, ...)

Arguments

Description

Converts a data frame of numeric data into a float tensor by calling as.matrix(). No input checks are performed

Usage

batchgetter_num(data, device, ...)

Arguments

Description

Creates an R6ClassGenerator inheriting from CallbackSet. Additionally performs checks such as that the stages are not accidentally misspelled. To create a TorchCallback use torch_callback().

In order for the resulting class to be cloneable, the private method ⁠$deep_clone()⁠ must be provided.

Usage

callback_set(
  classname,
  on_begin = NULL,
  on_end = NULL,
  on_exit = NULL,
  on_epoch_begin = NULL,
  on_before_valid = NULL,
  on_epoch_end = NULL,
  on_batch_begin = NULL,
  on_batch_end = NULL,
  on_after_backward = NULL,
  on_batch_valid_begin = NULL,
  on_batch_valid_end = NULL,
  on_valid_end = NULL,
  state_dict = NULL,
  load_state_dict = NULL,
  initialize = NULL,
  public = NULL,
  private = NULL,
  active = NULL,
  parent_env = parent.frame(),
  inherit = CallbackSet,
  lock_objects = FALSE
)

Arguments

Value

CallbackSet

See Also

Other Callback: TorchCallback, as_torch_callback(), as_torch_callbacks(), mlr3torch_callbacks, mlr_callback_set, mlr_callback_set.checkpoint, mlr_callback_set.progress, mlr_context_torch, t_clbk(), torch_callback()

Description

A data descriptor is a rather internal data structure used in the lazy_tensor data type. In essence it is an annotated torch::dataset and a preprocessing graph (consisting mosty of PipeOpModule operators). The additional meta data (e.g. pointer, shapes) allows to preprocess lazy_tensors in an mlr3pipelines::Graph just like any (non-lazy) data types. The preprocessing is applied when materialize() is called on the lazy_tensor.

To create a data descriptor, you can also use the as_data_descriptor() function.

Details

While it would be more natural to define this as an S3 class, we opted for an R6 class to avoid the usual trouble of serializing S3 objects. If each row contained a DataDescriptor as an S3 class, this would copy the object when serializing.

Public fields

Methods

Public methods

• DataDescriptor$new()

• DataDescriptor$print()

• DataDescriptor$clone()

Method new()

Creates a new instance of this R6 class.

Usage

DataDescriptor$new(
  dataset,
  dataset_shapes = NULL,
  graph = NULL,
  input_map = NULL,
  pointer = NULL,
  pointer_shape = NULL,
  pointer_shape_predict = NULL,
  clone_graph = TRUE
)

Arguments

Method print()

Prints the object

Usage

DataDescriptor$print(...)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

DataDescriptor$clone(deep = FALSE)

Arguments

See Also

ModelDescriptor, lazy_tensor

Examples

# Create a dataset
ds = dataset(
  initialize = function() self$x = torch_randn(10, 3, 3),
  .getitem = function(i) list(x = self$x[i, ]),
  .length = function() nrow(self$x)
)()
dd = DataDescriptor$new(ds, list(x = c(NA, 3, 3)))
dd
# is the same as using the converter:
as_data_descriptor(ds, list(x = c(NA, 3, 3)))

Description

Checks whether an object is a lazy tensor.

Usage

is_lazy_tensor(x)

Arguments

Description

Create a lazy tensor.

Usage

lazy_tensor(data_descriptor = NULL, ids = NULL)

Arguments

Examples

ds = dataset("example",
  initialize = function() self$iris = iris[, -5],
  .getitem = function(i) list(x = torch_tensor(as.numeric(self$iris[i, ]))),
  .length = function() nrow(self$iris)
)()
dd = as_data_descriptor(ds, list(x = c(NA, 4L)))
lt = as_lazy_tensor(dd)

Description

This will materialize a lazy_tensor() or a data.frame() / list() containing – among other things – lazy_tensor() columns. I.e. the data described in the underlying DataDescriptors is loaded for the indices in the lazy_tensor(), is preprocessed and then put unto the specified device. Because not all elements in a lazy tensor must have the same shape, a list of tensors is returned by default. If all elements have the same shape, these tensors can also be rbinded into a single tensor (parameter rbind).

Usage

materialize(x, device = "cpu", rbind = FALSE, ...)

## S3 method for class 'list'
materialize(x, device = "cpu", rbind = FALSE, cache = "auto", ...)

Arguments

Details

Materializing a lazy tensor consists of:

1. Loading the data from the internal dataset of the DataDescriptor.

2. Processing these batches in the preprocessing Graphs.

3. Returning the result of the PipeOp pointed to by the DataDescriptor (pointer).

With multiple lazy_tensor columns we can benefit from caching because: a) Output(s) from the dataset might be input to multiple graphs. b) Different lazy tensors might be outputs from the same graph.

For this reason it is possible to provide a cache environment. The hash key for a) is the hash of the indices and the dataset. The hash key for b) is the hash of the indices, dataset and preprocessing graph.

Value

(list() of lazy_tensors or a lazy_tensor)

Examples

lt1 = as_lazy_tensor(torch_randn(10, 3))
materialize(lt1, rbind = TRUE)
materialize(lt1, rbind = FALSE)
lt2 = as_lazy_tensor(torch_randn(10, 4))
d = data.table::data.table(lt1 = lt1, lt2 = lt2)
materialize(d, rbind = TRUE)
materialize(d, rbind = FALSE)

Description

This lazy data backend wraps a constructor that lazily creates another backend, e.g. by downloading (and caching) some data from the internet. This backend should be used, when some metadata of the backend is known in advance and should be accessible before downloading the actual data. When the backend is first constructed, it is verified that the provided metadata was correct, otherwise an informative error message is thrown. After the construction of the lazily constructed backend, calls like ⁠$data()⁠, ⁠$missings()⁠, ⁠$distinct()⁠, or ⁠$hash()⁠ are redirected to it.

Information that is available before the backend is constructed is:

• nrow - The number of rows (set as the length of the rownames).

• ncol - The number of columns (provided via the id column of col_info).

• colnames - The column names.

• rownames - The row names.

• col_info - The column information, which can be obtained via mlr3::col_info().

Beware that accessing the backend's hash also contructs the backend.

Note that while in most cases the data contains lazy_tensor columns, this is not necessary and the naming of this class has nothing to do with the lazy_tensor data type.

Important

When the constructor generates factor() variables it is important that the ordering of the levels in data corresponds to the ordering of the levels in the col_info argument.

Super class

mlr3::DataBackend -> DataBackendLazy

Active bindings

Methods

Public methods

• DataBackendLazy$new()

• DataBackendLazy$data()

• DataBackendLazy$head()

• DataBackendLazy$distinct()

• DataBackendLazy$missings()

• DataBackendLazy$print()

Method new()

Creates a new instance of this R6 class.

Usage

DataBackendLazy$new(constructor, rownames, col_info, primary_key)

Arguments

Method data()

Returns a slice of the data in the specified format. The rows must be addressed as vector of primary key values, columns must be referred to via column names. Queries for rows with no matching row id and queries for columns with no matching column name are silently ignored. Rows are guaranteed to be returned in the same order as rows, columns may be returned in an arbitrary order. Duplicated row ids result in duplicated rows, duplicated column names lead to an exception.

Accessing the data triggers the construction of the backend.

Usage

DataBackendLazy$data(rows, cols)

Arguments

Method head()

Retrieve the first n rows. This triggers the construction of the backend.

Usage

DataBackendLazy$head(n = 6L)

Arguments

Returns

data.table::data.table() of the first n rows.

Method distinct()

Returns a named list of vectors of distinct values for each column specified. If na_rm is TRUE, missing values are removed from the returned vectors of distinct values. Non-existing rows and columns are silently ignored.

This triggers the construction of the backend.

Usage

DataBackendLazy$distinct(rows, cols, na_rm = TRUE)

Arguments

Returns

Named list() of distinct values.

Method missings()

Returns the number of missing values per column in the specified slice of data. Non-existing rows and columns are silently ignored.

This triggers the construction of the backend.

Usage

DataBackendLazy$missings(rows, cols)

Arguments

Returns

Total of missing values per column (named numeric()).

Method print()

Printer.

Usage

DataBackendLazy$print()

Examples

# We first define a backend constructor
constructor = function(backend) {
  cat("Data is constructed!\n")
  DataBackendDataTable$new(
    data.table(x = rnorm(10), y = rnorm(10), row_id = 1:10),
    primary_key = "row_id"
  )
}

# to wrap this backend constructor in a lazy backend, we need to provide the correct metadata for it
column_info = data.table(
  id = c("x", "y", "row_id"),
  type = c("numeric", "numeric", "integer"),
  levels = list(NULL, NULL, NULL)
)
backend_lazy = DataBackendLazy$new(
  constructor = constructor,
  rownames = 1:10,
  col_info = column_info,
  primary_key = "row_id"
)

# Note that the constructor is not called for the calls below
# as they can be read from the metadata
backend_lazy$nrow
backend_lazy$rownames
backend_lazy$ncol
backend_lazy$colnames
col_info(backend_lazy)

# Only now the backend is constructed
backend_lazy$data(1, "x")
# Is the same as:
backend_lazy$backend$data(1, "x")

Description

Base class from which callbacks should inherit (see section Inheriting). A callback set is a collection of functions that are executed at different stages of the training loop. They can be used to gain more control over the training process of a neural network without having to write everything from scratch.

When used a in torch learner, the CallbackSet is wrapped in a TorchCallback. The latters parameter set represents the arguments of the CallbackSet's ⁠$initialize()⁠ method.

Inheriting

For each available stage (see section Stages) a public method ⁠$on_<stage>()⁠ can be defined. The evaluation context (a ContextTorch) can be accessed via self$ctx, which contains the current state of the training loop. This context is assigned at the beginning of the training loop and removed afterwards. Different stages of a callback can communicate with each other by assigning values to ⁠$self⁠.

State: To be able to store information in the ⁠$model⁠ slot of a LearnerTorch, callbacks support a state API. You can overload the ⁠$state_dict()⁠ public method to define what will be stored in ⁠learner$model$callbacks$<id>⁠ after training finishes. This then also requires to implement a ⁠$load_state_dict(state_dict)⁠ method that defines how to load a previously saved callback state into a different callback. Note that the ⁠$state_dict()⁠ should not include the parameter values that were used to initialize the callback.

For creating custom callbacks, the function torch_callback() is recommended, which creates a CallbackSet and then wraps it in a TorchCallback. To create a CallbackSet the convenience function callback_set() can be used. These functions perform checks such as that the stages are not accidentally misspelled.

Stages

• begin :: Run before the training loop begins.

• epoch_begin :: Run he beginning of each epoch.

• batch_begin :: Run before the forward call.

• after_backward :: Run after the backward call.

• batch_end :: Run after the optimizer step.

• batch_valid_begin :: Run before the forward call in the validation loop.

• batch_valid_end :: Run after the forward call in the validation loop.

• valid_end :: Run at the end of validation.

• epoch_end :: Run at the end of each epoch.

• end :: Run after last epoch.

• exit :: Run at last, using on.exit().

Terminate Training

If training is to be stopped, it is possible to set the field ⁠$terminate⁠ of ContextTorch. At the end of every epoch this field is checked and if it is TRUE, training stops. This can for example be used to implement custom early stopping.

Public fields

Active bindings

Methods

Public methods

• CallbackSet$print()

• CallbackSet$state_dict()

• CallbackSet$load_state_dict()

• CallbackSet$clone()

Method print()

Prints the object.

Usage

CallbackSet$print(...)

Arguments

Method state_dict()

Returns information that is kept in the the LearnerTorch's state after training. This information should be loadable into the callback using ⁠$load_state_dict()⁠ to be able to continue training. This returns NULL by default.

Usage

CallbackSet$state_dict()

Method load_state_dict()

Loads the state dict into the callback to continue training.

Usage

CallbackSet$load_state_dict(state_dict)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

CallbackSet$clone(deep = FALSE)

Arguments

See Also

Other Callback: TorchCallback, as_torch_callback(), as_torch_callbacks(), callback_set(), mlr3torch_callbacks, mlr_callback_set.checkpoint, mlr_callback_set.progress, mlr_context_torch, t_clbk(), torch_callback()

Description

Saves the optimizer and network states during training. The final network and optimizer are always stored.

Details

Saving the learner itself in the callback with a trained model is impossible, as the model slot is set after the last callback step is executed.

Super class

mlr3torch::CallbackSet -> CallbackSetCheckpoint

Methods

Public methods

• CallbackSetCheckpoint$new()

• CallbackSetCheckpoint$on_epoch_end()

• CallbackSetCheckpoint$on_batch_end()

• CallbackSetCheckpoint$on_exit()

• CallbackSetCheckpoint$clone()

Method new()

Creates a new instance of this R6 class.

Usage

CallbackSetCheckpoint$new(path, freq, freq_type = "epoch")

Arguments

Method on_epoch_end()

Saves the network and optimizer state dict. Does nothing if freq_type or freq are not met.

Usage

CallbackSetCheckpoint$on_epoch_end()

Method on_batch_end()

Saves the selected objects defined in save. Does nothing if freq_type or freq are not met.

Usage

CallbackSetCheckpoint$on_batch_end()

Method on_exit()

Saves the learner.

Usage

CallbackSetCheckpoint$on_exit()

Method clone()

The objects of this class are cloneable with this method.

Usage

CallbackSetCheckpoint$clone(deep = FALSE)

Arguments

See Also

Other Callback: TorchCallback, as_torch_callback(), as_torch_callbacks(), callback_set(), mlr3torch_callbacks, mlr_callback_set, mlr_callback_set.progress, mlr_context_torch, t_clbk(), torch_callback()

Description

Saves the training and validation history during training. The history is saved as a data.table in the ⁠$train⁠ and ⁠$valid⁠ slots. The first column is always epoch.

Super class

mlr3torch::CallbackSet -> CallbackSetHistory

Methods

Public methods

• CallbackSetHistory$on_begin()

• CallbackSetHistory$state_dict()

• CallbackSetHistory$load_state_dict()

• CallbackSetHistory$on_before_valid()

• CallbackSetHistory$on_epoch_end()

• CallbackSetHistory$clone()

Method on_begin()

Initializes lists where the train and validation metrics are stored.

Usage

CallbackSetHistory$on_begin()

Method state_dict()

Converts the lists to data.tables.

Usage

CallbackSetHistory$state_dict()

Method load_state_dict()

Sets the field ⁠$train⁠ and ⁠$valid⁠ to those contained in the state dict.

Usage

CallbackSetHistory$load_state_dict(state_dict)

Arguments

Method on_before_valid()

Add the latest training scores to the history.

Usage

CallbackSetHistory$on_before_valid()

Method on_epoch_end()

Add the latest validation scores to the history.

Usage

CallbackSetHistory$on_epoch_end()

Method clone()

The objects of this class are cloneable with this method.

Usage

CallbackSetHistory$clone(deep = FALSE)

Arguments

Description

Prints a progress bar and the metrics for training and validation.

Super class

mlr3torch::CallbackSet -> CallbackSetProgress

Methods

Public methods

• CallbackSetProgress$on_epoch_begin()

• CallbackSetProgress$on_batch_end()

• CallbackSetProgress$on_before_valid()

• CallbackSetProgress$on_batch_valid_end()

• CallbackSetProgress$on_epoch_end()

• CallbackSetProgress$clone()

Method on_epoch_begin()

Initializes the progress bar for training.

Usage

CallbackSetProgress$on_epoch_begin()

Method on_batch_end()

Increments the training progress bar.

Usage

CallbackSetProgress$on_batch_end()

Method on_before_valid()

Creates the progress bar for validation.

Usage

CallbackSetProgress$on_before_valid()

Method on_batch_valid_end()

Increments the validation progress bar.

Usage

CallbackSetProgress$on_batch_valid_end()

Method on_epoch_end()

Prints a summary of the training and validation process.

Usage

CallbackSetProgress$on_epoch_end()

Method clone()

The objects of this class are cloneable with this method.

Usage

CallbackSetProgress$clone(deep = FALSE)

Arguments

See Also

Other Callback: TorchCallback, as_torch_callback(), as_torch_callbacks(), callback_set(), mlr3torch_callbacks, mlr_callback_set, mlr_callback_set.checkpoint, mlr_context_torch, t_clbk(), torch_callback()

Description

Context for training a torch learner. This is the - mostly read-only - information callbacks have access to through the argument ctx. For more information on callbacks, see CallbackSet.

Public fields

Methods

Public methods

• ContextTorch$new()

• ContextTorch$clone()

Method new()

Creates a new instance of this R6 class.

Usage

ContextTorch$new(
  learner,
  task_train,
  task_valid = NULL,
  loader_train,
  loader_valid = NULL,
  measures_train = NULL,
  measures_valid = NULL,
  network,
  optimizer,
  loss_fn,
  total_epochs,
  prediction_encoder,
  eval_freq = 1L
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

ContextTorch$clone(deep = FALSE)

Arguments

See Also

Other Callback: TorchCallback, as_torch_callback(), as_torch_callbacks(), callback_set(), mlr3torch_callbacks, mlr_callback_set, mlr_callback_set.checkpoint, mlr_callback_set.progress, t_clbk(), torch_callback()

Description

This base class provides the basic functionality for training and prediction of a neural network. All torch learners should inherit from this class.

Validation

To specify the validation data, you can set the ⁠$validate⁠ field of the Learner, which can be set to:

• NULL: no validation

• ratio: only proportion 1 - ratio of the task is used for training and ratio is used for validation.

• "test" means that the "test" task of a resampling is used and is not possible when calling ⁠$train()⁠ manually.

• "predefined": This will use the predefined ⁠$internal_valid_task⁠ of a mlr3::Task.

This validation data can also be used for early stopping, see the description of the Learner's parameters.

Saving a Learner

In order to save a LearnerTorch for later usage, it is necessary to call the ⁠$marshal()⁠ method on the Learner before writing it to disk, as the object will otherwise not be saved correctly. After loading a marshaled LearnerTorch into R again, you then need to call ⁠$unmarshal()⁠ to transform it into a useable state.

Early Stopping and Tuning

In order to prevent overfitting, the LearnerTorch class allows to use early stopping via the patience and min_delta parameters, see the Learner's parameters. When tuning a LearnerTorch it is also possible to combine the explicit tuning via mlr3tuning and the LearnerTorch's internal tuning of the epochs via early stopping. To do so, you just need to include ⁠epochs = to_tune(upper = <upper>, internal = TRUE)⁠ in the search space, where ⁠<upper>⁠ is the maximally allowed number of epochs, and configure the early stopping.

Model

The Model is a list of class "learner_torch_model" with the following elements:

• network :: The trained network.

• optimizer :: The ⁠$state_dict()⁠ optimizer used to train the network.

• loss_fn :: The ⁠$state_dict()⁠ of the loss used to train the network.

• callbacks :: The callbacks used to train the network.

• seed :: The seed that was / is used for training and prediction.

• epochs :: How many epochs the model was trained for (early stopping).

• task_col_info :: A data.table() containing information about the train-task.

Parameters

General:

The parameters of the optimizer, loss and callbacks, prefixed with "opt.", "loss." and "cb.<callback id>." respectively, as well as:

• epochs :: integer(1)
  The number of epochs.

• device :: character(1)
  The device. One of "auto", "cpu", or "cuda" or other values defined in mlr_reflections$torch$devices. The value is initialized to "auto", which will select "cuda" if possible, then try "mps" and otherwise fall back to "cpu".

• num_threads :: integer(1)
  The number of threads for intraop pararallelization (if device is "cpu"). This value is initialized to 1.

• seed :: integer(1) or "random" or NULL
  The torch seed that is used during training and prediction. This value is initialized to "random", which means that a random seed will be sampled at the beginning of the training phase. This seed (either set or randomly sampled) is available via ⁠$model$seed⁠ after training and used during prediction. Note that by setting the seed during the training phase this will mean that by default (i.e. when seed is "random"), clones of the learner will use a different seed. If set to NULL, no seeding will be done.

Evaluation:

• measures_train :: Measure or list() of Measures.
  Measures to be evaluated during training.

• measures_valid :: Measure or list() of Measures.
  Measures to be evaluated during validation.

• eval_freq :: integer(1)
  How often the train / validation predictions are evaluated using measures_train / measures_valid. This is initialized to 1. Note that the final model is always evaluated.

Early Stopping:

• patience :: integer(1)
  This activates early stopping using the validation scores. If the performance of a model does not improve for patience evaluation steps, training is ended. Note that the final model is stored in the learner, not the best model. This is initialized to 0, which means no early stopping. The first entry from measures_valid is used as the metric. This also requires to specify the ⁠$validate⁠ field of the Learner, as well as measures_valid.

• min_delta :: double(1)
  The minimum improvement threshold (>) for early stopping. Is initialized to 0.

Dataloader:

• batch_size :: integer(1)
  The batch size (required).

• shuffle :: logical(1)
  Whether to shuffle the instances in the dataset. Default is FALSE. This does not impact validation.

• sampler :: torch::sampler
  Object that defines how the dataloader draw samples.

• batch_sampler :: torch::sampler
  Object that defines how the dataloader draws batches.

• num_workers :: integer(1)
  The number of workers for data loading (batches are loaded in parallel). The default is 0, which means that data will be loaded in the main process.

• collate_fn :: function
  How to merge a list of samples to form a batch.

• pin_memory :: logical(1)
  Whether the dataloader copies tensors into CUDA pinned memory before returning them.

• drop_last :: logical(1)
  Whether to drop the last training batch in each epoch during training. Default is FALSE.

• timeout :: numeric(1)
  The timeout value for collecting a batch from workers. Negative values mean no timeout and the default is -1.

• worker_init_fn :: ⁠function(id)⁠
  A function that receives the worker id (in ⁠[1, num_workers]⁠) and is exectued after seeding on the worker but before data loading.

• worker_globals :: list() | character()
  When loading data in parallel, this allows to export globals to the workers. If this is a character vector, the objects in the global environment with those names are copied to the workers.

• worker_packages :: character()
  Which packages to load on the workers.

Also see torch::dataloder for more information.

Inheriting

There are no seperate classes for classification and regression to inherit from. Instead, the task_type must be specified as a construction argument. Currently, only classification and regression are supported.

When inheriting from this class, one should overload two private methods:

• .network(task, param_vals)
  (Task, list()) -> nn_module
  Construct a torch::nn_module object for the given task and parameter values, i.e. the neural network that is trained by the learner. For classification, the output of this network are expected to be the scores before the application of the final softmax layer.

• .dataset(task, param_vals)
  (Task, list()) -> torch::dataset
  Create the dataset for the task. Must respect the parameter value of the device. Moreover, one needs to pay attention respect the row ids of the provided task.

It is also possible to overwrite the private .dataloader() method instead of the .dataset() method. Per default, a dataloader is constructed using the output from the .dataset() method. However, this should respect the dataloader parameters from the ParamSet.

• .dataloader(task, param_vals)
  (Task, list()) -> torch::dataloader
  Create a dataloader from the task. Needs to respect at least batch_size and shuffle (otherwise predictions can be permuted).

To change the predict types, the private .encode_prediction() method can be overwritten:

• .encode_prediction(predict_tensor, task, param_vals)
  (torch_tensor, Task, list()) -> list()
  Take in the raw predictions from self$network (predict_tensor) and encode them into a format that can be converted to valid mlr3 predictions using mlr3::as_prediction_data(). This method must take self$predict_type into account.

While it is possible to add parameters by specifying the param_set construction argument, it is currently not possible to remove existing parameters, i.e. those listed in section Parameters. None of the parameters provided in param_set can have an id that starts with "loss.", ⁠"opt.", or ⁠"cb."', as these are preserved for the dynamically constructed parameters of the optimizer, the loss function, and the callbacks.

To perform additional input checks on the task, the private .verify_train_task(task, param_vals) and .verify_predict_task(task, param_vals) can be overwritten.

For learners that have other construction arguments that should change the hash of a learner, it is required to implement the private ⁠$.additional_phash_input()⁠.

Super class

mlr3::Learner -> LearnerTorch

Active bindings

Methods

Public methods

• LearnerTorch$new()

• LearnerTorch$format()

• LearnerTorch$print()

• LearnerTorch$marshal()

• LearnerTorch$unmarshal()

• LearnerTorch$dataset()

• LearnerTorch$clone()

Method new()

Creates a new instance of this R6 class.

Usage

LearnerTorch$new(
  id,
  task_type,
  param_set,
  properties,
  man,
  label,
  feature_types,
  optimizer = NULL,
  loss = NULL,
  packages = character(),
  predict_types = NULL,
  callbacks = list()
)

Arguments

Method format()

Helper for print outputs.

Usage

LearnerTorch$format(...)

Arguments

Method print()

Prints the object.

Usage

LearnerTorch$print(...)

Arguments

Method marshal()

Marshal the learner.

Usage

LearnerTorch$marshal(...)

Arguments

Returns

self

Method unmarshal()

Unmarshal the learner.

Usage

LearnerTorch$unmarshal(...)

Arguments

Returns

self

Method dataset()

Create the dataset for a task.

Usage

LearnerTorch$dataset(task)

Arguments

Returns

dataset

Method clone()

The objects of this class are cloneable with this method.

Usage

LearnerTorch$clone(deep = FALSE)

Arguments

See Also

Other Learner: mlr_learners.mlp, mlr_learners.tab_resnet, mlr_learners.torch_featureless, mlr_learners_torch_image, mlr_learners_torch_model

Description

Base Class for Image Learners. The features are assumed to be a single lazy_tensor column in RGB format.

Parameters

Parameters include those inherited from LearnerTorch and the param_set construction argument.

Super classes

mlr3::Learner -> mlr3torch::LearnerTorch -> LearnerTorchImage

Methods

Public methods

• LearnerTorchImage$new()

• LearnerTorchImage$clone()

Method new()

Creates a new instance of this R6 class.

Usage

LearnerTorchImage$new(
  id,
  task_type,
  param_set = ps(),
  label,
  optimizer = NULL,
  loss = NULL,
  callbacks = list(),
  packages = c("torchvision", "magick"),
  man,
  properties = NULL,
  predict_types = NULL
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

LearnerTorchImage$clone(deep = FALSE)

Arguments

See Also

Other Learner: mlr_learners.mlp, mlr_learners.tab_resnet, mlr_learners.torch_featureless, mlr_learners_torch, mlr_learners_torch_model

Description

Create a torch learner from an instantiated nn_module(). For classification, the output of the network must be the scores (before the softmax).

Parameters

See LearnerTorch

Super classes

mlr3::Learner -> mlr3torch::LearnerTorch -> LearnerTorchModel

Active bindings

Methods

Public methods

• LearnerTorchModel$new()

• LearnerTorchModel$clone()

Method new()

Creates a new instance of this R6 class.

Usage

LearnerTorchModel$new(
  network = NULL,
  ingress_tokens = NULL,
  task_type,
  properties = NULL,
  optimizer = NULL,
  loss = NULL,
  callbacks = list(),
  packages = character(0),
  feature_types = NULL
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

LearnerTorchModel$clone(deep = FALSE)

Arguments

See Also

Other Learner: mlr_learners.mlp, mlr_learners.tab_resnet, mlr_learners.torch_featureless, mlr_learners_torch, mlr_learners_torch_image

Other Graph Network: ModelDescriptor(), TorchIngressToken(), mlr_pipeops_module, mlr_pipeops_torch, mlr_pipeops_torch_ingress, mlr_pipeops_torch_ingress_categ, mlr_pipeops_torch_ingress_ltnsr, mlr_pipeops_torch_ingress_num, model_descriptor_to_learner(), model_descriptor_to_module(), model_descriptor_union(), nn_graph()

Examples

# We show the learner using a classification task

# The iris task has 4 features and 3 classes
network = nn_linear(4, 3)
task = tsk("iris")

# This defines the dataloader.
# It loads all 4 features, which are also numeric.
# The shape is (NA, 4) because the batch dimension is generally NA
ingress_tokens = list(
  input = TorchIngressToken(task$feature_names, batchgetter_num, c(NA, 4))
)

# Creating the learner and setting required parameters
learner = lrn("classif.torch_model",
  network = network,
  ingress_tokens = ingress_tokens,
  batch_size = 16,
  epochs = 1,
  device = "cpu"
)

# A simple train-predict
ids = partition(task)
learner$train(task, ids$train)
learner$predict(task, ids$test)

Description

Fully connected feed forward network with dropout after each activation function. The features can either be a single lazy_tensor or one or more numeric columns (but not both).

Dictionary

This Learner can be instantiated using the sugar function lrn():

lrn("classif.mlp", ...)
lrn("regr.mlp", ...)

Properties

• Supported task types: 'classif', 'regr'

• Predict Types:

  • classif: 'response', 'prob'

  • regr: 'response'

• Feature Types: “integer”, “numeric”, “lazy_tensor”

• Required Packages: mlr3, mlr3torch, torch

Parameters

Parameters from LearnerTorch, as well as:

• activation :: ⁠[nn_module]⁠
  The activation function. Is initialized to nn_relu.

• activation_args :: named list()
  A named list with initialization arguments for the activation function. This is intialized to an empty list.

• neurons :: integer()
  The number of neurons per hidden layer. By default there is no hidden layer. Setting this to c(10, 20) would have a the first hidden layer with 10 neurons and the second with 20.

• p :: numeric(1)
  The dropout probability. Is initialized to 0.5.

• shape :: integer() or NULL
  The input shape of length 2, e.g. c(NA, 5). Only needs to be present when there is a lazy tensor input with unknown shape (NULL). Otherwise the input shape is inferred from the number of numeric features.

Super classes

mlr3::Learner -> mlr3torch::LearnerTorch -> LearnerTorchMLP

Methods

Public methods

• LearnerTorchMLP$new()

• LearnerTorchMLP$clone()

Method new()

Creates a new instance of this R6 class.

Usage

LearnerTorchMLP$new(
  task_type,
  optimizer = NULL,
  loss = NULL,
  callbacks = list()
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

LearnerTorchMLP$clone(deep = FALSE)

Arguments

See Also

Other Learner: mlr_learners.tab_resnet, mlr_learners.torch_featureless, mlr_learners_torch, mlr_learners_torch_image, mlr_learners_torch_model

Examples

# Define the Learner and set parameter values
learner = lrn("classif.mlp")
learner$param_set$set_values(
  epochs = 1, batch_size = 16, device = "cpu",
  neurons = 10
)

# Define a Task
task = tsk("iris")

# Create train and test set
ids = partition(task)

# Train the learner on the training ids
learner$train(task, row_ids = ids$train)

# Make predictions for the test rows
predictions = learner$predict(task, row_ids = ids$test)

# Score the predictions
predictions$score()

Description

Tabular resnet.

Dictionary

This Learner can be instantiated using the sugar function lrn():

lrn("classif.tab_resnet", ...)
lrn("regr.tab_resnet", ...)

Properties

• Supported task types: 'classif', 'regr'

• Predict Types:

  • classif: 'response', 'prob'

  • regr: 'response'

• Feature Types: “integer”, “numeric”

• Required Packages: mlr3, mlr3torch, torch

Parameters

Parameters from LearnerTorch, as well as:

• n_blocks :: integer(1)
  The number of blocks.

• d_block :: integer(1)
  The input and output dimension of a block.

• d_hidden :: integer(1)
  The latent dimension of a block.

• d_hidden_multiplier :: integer(1)
  Alternative way to specify the latent dimension as d_block * d_hidden_multiplier.

• dropout1 :: numeric(1)
  First dropout ratio.

• dropout2 :: numeric(1)
  Second dropout ratio.

Super classes

mlr3::Learner -> mlr3torch::LearnerTorch -> LearnerTorchTabResNet

Methods

Public methods

• LearnerTorchTabResNet$new()

• LearnerTorchTabResNet$clone()

Method new()

Creates a new instance of this R6 class.

Usage

LearnerTorchTabResNet$new(
  task_type,
  optimizer = NULL,
  loss = NULL,
  callbacks = list()
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

LearnerTorchTabResNet$clone(deep = FALSE)

Arguments

References

Gorishniy Y, Rubachev I, Khrulkov V, Babenko A (2021). “Revisiting Deep Learning for Tabular Data.” arXiv, 2106.11959.

See Also

Other Learner: mlr_learners.mlp, mlr_learners.torch_featureless, mlr_learners_torch, mlr_learners_torch_image, mlr_learners_torch_model

Examples

# Define the Learner and set parameter values
learner = lrn("classif.tab_resnet")
learner$param_set$set_values(
  epochs = 1, batch_size = 16, device = "cpu",
  n_blocks = 2, d_block = 10, d_hidden = 20, dropout1 = 0.3, dropout2 = 0.3
)

# Define a Task
task = tsk("iris")

# Create train and test set
ids = partition(task)

# Train the learner on the training ids
learner$train(task, row_ids = ids$train)

# Make predictions for the test rows
predictions = learner$predict(task, row_ids = ids$test)

# Score the predictions
predictions$score()

Description

Featureless torch learner. Output is a constant weight that is learned during training. For classification, this should (asymptoptically) result in a majority class prediction when using the standard cross-entropy loss. For regression, this should result in the median for L1 loss and in the mean for L2 loss.

Dictionary

This Learner can be instantiated using the sugar function lrn():

lrn("classif.torch_featureless", ...)
lrn("regr.torch_featureless", ...)

Properties

• Supported task types: 'classif', 'regr'

• Predict Types:

  • classif: 'response', 'prob'

  • regr: 'response'

• Feature Types: “logical”, “integer”, “numeric”, “character”, “factor”, “ordered”, “POSIXct”, “lazy_tensor”

• Required Packages: mlr3, mlr3torch, torch

Parameters

Only those from LearnerTorch.

Super classes

mlr3::Learner -> mlr3torch::LearnerTorch -> LearnerTorchFeatureless

Methods

Public methods

• LearnerTorchFeatureless$new()

• LearnerTorchFeatureless$clone()

Method new()

Creates a new instance of this R6 class.

Usage

LearnerTorchFeatureless$new(
  task_type,
  optimizer = NULL,
  loss = NULL,
  callbacks = list()
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

LearnerTorchFeatureless$clone(deep = FALSE)

Arguments

See Also

Other Learner: mlr_learners.mlp, mlr_learners.tab_resnet, mlr_learners_torch, mlr_learners_torch_image, mlr_learners_torch_model

Examples

# Define the Learner and set parameter values
learner = lrn("classif.torch_featureless")
learner$param_set$set_values(
  epochs = 1, batch_size = 16, device = "cpu"
  
)

# Define a Task
task = tsk("iris")

# Create train and test set
ids = partition(task)

# Train the learner on the training ids
learner$train(task, row_ids = ids$train)

# Make predictions for the test rows
predictions = learner$predict(task, row_ids = ids$test)

# Score the predictions
predictions$score()

Description

Classic image classification networks from torchvision.

Parameters

Parameters from LearnerTorchImage and

• pretrained :: logical(1)
  Whether to use the pretrained model. The final linear layer will be replaced with a new nn_linear with the number of classes inferred from the Task.

Properties

• Supported task types: "classif"

• Predict Types: "response" and "prob"

• Feature Types: "lazy_tensor"

• Required packages: "mlr3torch", "torch", "torchvision"

Super classes

mlr3::Learner -> mlr3torch::LearnerTorch -> mlr3torch::LearnerTorchImage -> LearnerTorchVision

Methods

Public methods

• LearnerTorchVision$new()

• LearnerTorchVision$clone()

Method new()

Creates a new instance of this R6 class.

Usage

LearnerTorchVision$new(
  name,
  module_generator,
  label,
  optimizer = NULL,
  loss = NULL,
  callbacks = list()
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

LearnerTorchVision$clone(deep = FALSE)

Arguments

Description

Calls torchvision::transform_center_crop, see there for more information on the parameters. The preprocessing is applied row wise (no batch dimension).

Format

R6Class inheriting from PipeOpTaskPreprocTorch.

Parameters

Description

Calls torchvision::transform_color_jitter, see there for more information on the parameters. The preprocessing is applied row wise (no batch dimension).

Format

R6Class inheriting from PipeOpTaskPreprocTorch.

Parameters

Description

Calls torchvision::transform_crop, see there for more information on the parameters. The preprocessing is applied row wise (no batch dimension).

Format

R6Class inheriting from PipeOpTaskPreprocTorch.

Parameters

Description

Calls torchvision::transform_hflip, see there for more information on the parameters. The preprocessing is applied row wise (no batch dimension).

Format

R6Class inheriting from PipeOpTaskPreprocTorch.

Parameters

Description

Calls torchvision::transform_random_affine, see there for more information on the parameters. The preprocessing is applied row wise (no batch dimension).

Format

R6Class inheriting from PipeOpTaskPreprocTorch.

Parameters

Description

Calls torchvision::transform_random_choice, see there for more information on the parameters. The preprocessing is applied row wise (no batch dimension).

Format

R6Class inheriting from PipeOpTaskPreprocTorch.

Parameters

Description

Calls torchvision::transform_random_crop, see there for more information on the parameters. The preprocessing is applied row wise (no batch dimension).

Format

R6Class inheriting from PipeOpTaskPreprocTorch.

Parameters

Description

Calls torchvision::transform_random_horizontal_flip, see there for more information on the parameters. The preprocessing is applied row wise (no batch dimension).

Format

R6Class inheriting from PipeOpTaskPreprocTorch.

Parameters

Description

Calls torchvision::transform_random_order, see there for more information on the parameters. The preprocessing is applied row wise (no batch dimension).

Format

R6Class inheriting from PipeOpTaskPreprocTorch.

Parameters

Description

Calls torchvision::transform_random_resized_crop, see there for more information on the parameters. The preprocessing is applied row wise (no batch dimension).

Format

R6Class inheriting from PipeOpTaskPreprocTorch.

Parameters

Description

Calls torchvision::transform_random_vertical_flip, see there for more information on the parameters. The preprocessing is applied row wise (no batch dimension).

Format

R6Class inheriting from PipeOpTaskPreprocTorch.

Parameters

Description

Calls torchvision::transform_resized_crop, see there for more information on the parameters. The preprocessing is applied row wise (no batch dimension).

Format

R6Class inheriting from PipeOpTaskPreprocTorch.

Parameters

Description

Calls torchvision::transform_rotate, see there for more information on the parameters. The preprocessing is applied row wise (no batch dimension).

Format

R6Class inheriting from PipeOpTaskPreprocTorch.

Parameters

Description

Calls torchvision::transform_vflip, see there for more information on the parameters. The preprocessing is applied row wise (no batch dimension).

Format

R6Class inheriting from PipeOpTaskPreprocTorch.

Parameters

Description

PipeOpModule wraps an nn_module or function that is being called during the train phase of this mlr3pipelines::PipeOp. By doing so, this allows to assemble PipeOpModules in a computational mlr3pipelines::Graph that represents either a neural network or a preprocessing graph of a lazy_tensor. In most cases it is easier to create such a network by creating a graph that generates this graph.

In most cases it is easier to create such a network by creating a structurally related graph consisting of nodes of class PipeOpTorchIngress and PipeOpTorch. This graph will then generate the graph consisting of PipeOpModules as part of the ModelDescriptor.

Input and Output Channels

The number and names of the input and output channels can be set during construction. They input and output "torch_tensor" during training, and NULL during prediction as the prediction phase currently serves no meaningful purpose.

State

The state is the value calculated by the public method shapes_out().

Parameters

No parameters.

Internals

During training, the wrapped nn_module / function is called with the provided inputs in the order in which the channels are defined. Arguments are not matched by name.

Super class

mlr3pipelines::PipeOp -> PipeOpModule

Public fields

Methods

Public methods

• PipeOpModule$new()

• PipeOpModule$clone()

Method new()

Creates a new instance of this R6 class.

Usage

PipeOpModule$new(
  id = "module",
  module = nn_identity(),
  inname = "input",
  outname = "output",
  param_vals = list(),
  packages = character(0)
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

PipeOpModule$clone(deep = FALSE)

Arguments

See Also

Other Graph Network: ModelDescriptor(), TorchIngressToken(), mlr_learners_torch_model, mlr_pipeops_torch, mlr_pipeops_torch_ingress, mlr_pipeops_torch_ingress_categ, mlr_pipeops_torch_ingress_ltnsr, mlr_pipeops_torch_ingress_num, model_descriptor_to_learner(), model_descriptor_to_module(), model_descriptor_union(), nn_graph()

Other PipeOp: mlr_pipeops_torch_callbacks, mlr_pipeops_torch_optimizer

Examples

## creating an PipeOpModule manually

# one input and output channel
po_module = po("module",
  id = "linear",
  module = torch::nn_linear(10, 20),
  inname = "input",
  outname = "output"
)
x = torch::torch_randn(16, 10)
# This calls the forward function of the wrapped module.
y = po_module$train(list(input = x))
str(y)

# multiple input and output channels
nn_custom = torch::nn_module("nn_custom",
  initialize = function(in_features, out_features) {
    self$lin1 = torch::nn_linear(in_features, out_features)
    self$lin2 = torch::nn_linear(in_features, out_features)
  },
  forward = function(x, z) {
    list(out1 = self$lin1(x), out2 = torch::nnf_relu(self$lin2(z)))
  }
)

module = nn_custom(3, 2)
po_module = po("module",
  id = "custom",
  module = module,
  inname = c("x", "z"),
  outname = c("out1", "out2")
)
x = torch::torch_randn(1, 3)
z = torch::torch_randn(1, 3)
out = po_module$train(list(x = x, z = z))
str(out)

# How such a PipeOpModule is usually generated
graph = po("torch_ingress_num") %>>% po("nn_linear", out_features = 10L)
result = graph$train(tsk("iris"))
# The PipeOpTorchLinear generates a PipeOpModule and adds it to a new (module) graph
result[[1]]$graph
linear_module = result[[1L]]$graph$pipeops$nn_linear
linear_module
formalArgs(linear_module$module)
linear_module$input$name

# Constructing a PipeOpModule using a simple function
po_add1 = po("module",
  id = "add_one",
  module = function(x) x + 1
)
input = list(torch_tensor(1))
po_add1$train(input)$output

Description

Applies a 1D adaptive average pooling over an input signal composed of several input planes.

Input and Output Channels

One input channel called "input" and one output channel called "output". For an explanation see PipeOpTorch.

State

The state is the value calculated by the public method ⁠$shapes_out()⁠.

Credit

Part of this documentation have been copied or adapted from the documentation of torch.

Parameters

• kernel_size :: (integer())
  The size of the window. Can be a single number or a vector.

• stride :: integer()
  The stride of the window. Can be a single number or a vector. Default: kernel_size.

• padding :: integer()
  Implicit zero paddings on both sides of the input. Can be a single number or a vector. Default: 0.

• ceil_mode :: integer()
  When TRUE, will use ceil instead of floor to compute the output shape. Default: FALSE.

• count_include_pad :: logical(1)
  When TRUE, will include the zero-padding in the averaging calculation. Default: TRUE.

• divisor_override :: logical(1)
  If specified, it will be used as divisor, otherwise size of the pooling region will be used. Default: NULL. Only available for dimension greater than 1.

Internals

Calls nn_avg_pool1d() during training.

Super classes

mlr3pipelines::PipeOp -> mlr3torch::PipeOpTorch -> mlr3torch::PipeOpTorchAvgPool -> PipeOpTorchAvgPool1D

Methods

Public methods

• PipeOpTorchAvgPool1D$new()

• PipeOpTorchAvgPool1D$clone()

Method new()

Creates a new instance of this R6 class.

Usage

PipeOpTorchAvgPool1D$new(id = "nn_avg_pool1d", param_vals = list())

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

PipeOpTorchAvgPool1D$clone(deep = FALSE)

Arguments

See Also

Other PipeOps: mlr_pipeops_nn_avg_pool2d, mlr_pipeops_nn_avg_pool3d, mlr_pipeops_nn_batch_norm1d, mlr_pipeops_nn_batch_norm2d, mlr_pipeops_nn_batch_norm3d, mlr_pipeops_nn_block, mlr_pipeops_nn_celu, mlr_pipeops_nn_conv1d, mlr_pipeops_nn_conv2d, mlr_pipeops_nn_conv3d, mlr_pipeops_nn_conv_transpose1d, mlr_pipeops_nn_conv_transpose2d, mlr_pipeops_nn_conv_transpose3d, mlr_pipeops_nn_dropout, mlr_pipeops_nn_elu, mlr_pipeops_nn_flatten, mlr_pipeops_nn_gelu, mlr_pipeops_nn_glu, mlr_pipeops_nn_hardshrink, mlr_pipeops_nn_hardsigmoid, mlr_pipeops_nn_hardtanh, mlr_pipeops_nn_head, mlr_pipeops_nn_layer_norm, mlr_pipeops_nn_leaky_relu, mlr_pipeops_nn_linear, mlr_pipeops_nn_log_sigmoid, mlr_pipeops_nn_max_pool1d, mlr_pipeops_nn_max_pool2d, mlr_pipeops_nn_max_pool3d, mlr_pipeops_nn_merge, mlr_pipeops_nn_merge_cat, mlr_pipeops_nn_merge_prod, mlr_pipeops_nn_merge_sum, mlr_pipeops_nn_prelu, mlr_pipeops_nn_relu, mlr_pipeops_nn_relu6, mlr_pipeops_nn_reshape, mlr_pipeops_nn_rrelu, mlr_pipeops_nn_selu, mlr_pipeops_nn_sigmoid, mlr_pipeops_nn_softmax, mlr_pipeops_nn_softplus, mlr_pipeops_nn_softshrink, mlr_pipeops_nn_softsign, mlr_pipeops_nn_squeeze, mlr_pipeops_nn_tanh, mlr_pipeops_nn_tanhshrink, mlr_pipeops_nn_threshold, mlr_pipeops_nn_unsqueeze, mlr_pipeops_torch_ingress, mlr_pipeops_torch_ingress_categ, mlr_pipeops_torch_ingress_ltnsr, mlr_pipeops_torch_ingress_num, mlr_pipeops_torch_loss, mlr_pipeops_torch_model, mlr_pipeops_torch_model_classif, mlr_pipeops_torch_model_regr

Examples

# Construct the PipeOp
pipeop = po("nn_avg_pool1d")
pipeop
# The available parameters
pipeop$param_set

Description

Applies a 2D adaptive average pooling over an input signal composed of several input planes.

Input and Output Channels

One input channel called "input" and one output channel called "output". For an explanation see PipeOpTorch.

State

The state is the value calculated by the public method ⁠$shapes_out()⁠.

Credit

Part of this documentation have been copied or adapted from the documentation of torch.

Internals

Calls nn_avg_pool2d() during training.

Parameters

• kernel_size :: (integer())
  The size of the window. Can be a single number or a vector.

• stride :: integer()
  The stride of the window. Can be a single number or a vector. Default: kernel_size.

• padding :: integer()
  Implicit zero paddings on both sides of the input. Can be a single number or a vector. Default: 0.

• ceil_mode :: integer()
  When TRUE, will use ceil instead of floor to compute the output shape. Default: FALSE.

• count_include_pad :: logical(1)
  When TRUE, will include the zero-padding in the averaging calculation. Default: TRUE.

• divisor_override :: logical(1)
  If specified, it will be used as divisor, otherwise size of the pooling region will be used. Default: NULL. Only available for dimension greater than 1.

Super classes

mlr3pipelines::PipeOp -> mlr3torch::PipeOpTorch -> mlr3torch::PipeOpTorchAvgPool -> PipeOpTorchAvgPool2D

Methods

Public methods

• PipeOpTorchAvgPool2D$new()

• PipeOpTorchAvgPool2D$clone()

Method new()

Creates a new instance of this R6 class.

Usage

PipeOpTorchAvgPool2D$new(id = "nn_avg_pool2d", param_vals = list())

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

PipeOpTorchAvgPool2D$clone(deep = FALSE)

Arguments

See Also

Other PipeOps: mlr_pipeops_nn_avg_pool1d, mlr_pipeops_nn_avg_pool3d, mlr_pipeops_nn_batch_norm1d, mlr_pipeops_nn_batch_norm2d, mlr_pipeops_nn_batch_norm3d, mlr_pipeops_nn_block, mlr_pipeops_nn_celu, mlr_pipeops_nn_conv1d, mlr_pipeops_nn_conv2d, mlr_pipeops_nn_conv3d, mlr_pipeops_nn_conv_transpose1d, mlr_pipeops_nn_conv_transpose2d, mlr_pipeops_nn_conv_transpose3d, mlr_pipeops_nn_dropout, mlr_pipeops_nn_elu, mlr_pipeops_nn_flatten, mlr_pipeops_nn_gelu, mlr_pipeops_nn_glu, mlr_pipeops_nn_hardshrink, mlr_pipeops_nn_hardsigmoid, mlr_pipeops_nn_hardtanh, mlr_pipeops_nn_head, mlr_pipeops_nn_layer_norm, mlr_pipeops_nn_leaky_relu, mlr_pipeops_nn_linear, mlr_pipeops_nn_log_sigmoid, mlr_pipeops_nn_max_pool1d, mlr_pipeops_nn_max_pool2d, mlr_pipeops_nn_max_pool3d, mlr_pipeops_nn_merge, mlr_pipeops_nn_merge_cat, mlr_pipeops_nn_merge_prod, mlr_pipeops_nn_merge_sum, mlr_pipeops_nn_prelu, mlr_pipeops_nn_relu, mlr_pipeops_nn_relu6, mlr_pipeops_nn_reshape, mlr_pipeops_nn_rrelu, mlr_pipeops_nn_selu, mlr_pipeops_nn_sigmoid, mlr_pipeops_nn_softmax, mlr_pipeops_nn_softplus, mlr_pipeops_nn_softshrink, mlr_pipeops_nn_softsign, mlr_pipeops_nn_squeeze, mlr_pipeops_nn_tanh, mlr_pipeops_nn_tanhshrink, mlr_pipeops_nn_threshold, mlr_pipeops_nn_unsqueeze, mlr_pipeops_torch_ingress, mlr_pipeops_torch_ingress_categ, mlr_pipeops_torch_ingress_ltnsr, mlr_pipeops_torch_ingress_num, mlr_pipeops_torch_loss, mlr_pipeops_torch_model, mlr_pipeops_torch_model_classif, mlr_pipeops_torch_model_regr

Examples

# Construct the PipeOp
pipeop = po("nn_avg_pool2d")
pipeop
# The available parameters
pipeop$param_set

Description

Applies a 3D adaptive average pooling over an input signal composed of several input planes.

Input and Output Channels

One input channel called "input" and one output channel called "output". For an explanation see PipeOpTorch.

State

The state is the value calculated by the public method ⁠$shapes_out()⁠.

Credit

Part of this documentation have been copied or adapted from the documentation of torch.

Internals

Calls nn_avg_pool3d() during training.

Parameters

• kernel_size :: (integer())
  The size of the window. Can be a single number or a vector.

• stride :: integer()
  The stride of the window. Can be a single number or a vector. Default: kernel_size.

• padding :: integer()
  Implicit zero paddings on both sides of the input. Can be a single number or a vector. Default: 0.

• ceil_mode :: integer()
  When TRUE, will use ceil instead of floor to compute the output shape. Default: FALSE.

• count_include_pad :: logical(1)
  When TRUE, will include the zero-padding in the averaging calculation. Default: TRUE.

• divisor_override :: logical(1)
  If specified, it will be used as divisor, otherwise size of the pooling region will be used. Default: NULL. Only available for dimension greater than 1.

Super classes

mlr3pipelines::PipeOp -> mlr3torch::PipeOpTorch -> mlr3torch::PipeOpTorchAvgPool -> PipeOpTorchAvgPool3D

Methods

Public methods

• PipeOpTorchAvgPool3D$new()

• PipeOpTorchAvgPool3D$clone()

Method new()

Creates a new instance of this R6 class.

Usage

PipeOpTorchAvgPool3D$new(id = "nn_avg_pool3d", param_vals = list())

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

PipeOpTorchAvgPool3D$clone(deep = FALSE)

Arguments

See Also

Other PipeOps: mlr_pipeops_nn_avg_pool1d, mlr_pipeops_nn_avg_pool2d, mlr_pipeops_nn_batch_norm1d, mlr_pipeops_nn_batch_norm2d, mlr_pipeops_nn_batch_norm3d, mlr_pipeops_nn_block, mlr_pipeops_nn_celu, mlr_pipeops_nn_conv1d, mlr_pipeops_nn_conv2d, mlr_pipeops_nn_conv3d, mlr_pipeops_nn_conv_transpose1d, mlr_pipeops_nn_conv_transpose2d, mlr_pipeops_nn_conv_transpose3d, mlr_pipeops_nn_dropout, mlr_pipeops_nn_elu, mlr_pipeops_nn_flatten, mlr_pipeops_nn_gelu, mlr_pipeops_nn_glu, mlr_pipeops_nn_hardshrink, mlr_pipeops_nn_hardsigmoid, mlr_pipeops_nn_hardtanh, mlr_pipeops_nn_head, mlr_pipeops_nn_layer_norm, mlr_pipeops_nn_leaky_relu, mlr_pipeops_nn_linear, mlr_pipeops_nn_log_sigmoid, mlr_pipeops_nn_max_pool1d, mlr_pipeops_nn_max_pool2d, mlr_pipeops_nn_max_pool3d, mlr_pipeops_nn_merge, mlr_pipeops_nn_merge_cat, mlr_pipeops_nn_merge_prod, mlr_pipeops_nn_merge_sum, mlr_pipeops_nn_prelu, mlr_pipeops_nn_relu, mlr_pipeops_nn_relu6, mlr_pipeops_nn_reshape, mlr_pipeops_nn_rrelu, mlr_pipeops_nn_selu, mlr_pipeops_nn_sigmoid, mlr_pipeops_nn_softmax, mlr_pipeops_nn_softplus, mlr_pipeops_nn_softshrink, mlr_pipeops_nn_softsign, mlr_pipeops_nn_squeeze, mlr_pipeops_nn_tanh, mlr_pipeops_nn_tanhshrink, mlr_pipeops_nn_threshold, mlr_pipeops_nn_unsqueeze, mlr_pipeops_torch_ingress, mlr_pipeops_torch_ingress_categ, mlr_pipeops_torch_ingress_ltnsr, mlr_pipeops_torch_ingress_num, mlr_pipeops_torch_loss, mlr_pipeops_torch_model, mlr_pipeops_torch_model_classif, mlr_pipeops_torch_model_regr

Examples

# Construct the PipeOp
pipeop = po("nn_avg_pool3d")
pipeop
# The available parameters
pipeop$param_set

Description

Applies Batch Normalization for each channel across a batch of data.

Input and Output Channels

One input channel called "input" and one output channel called "output". For an explanation see PipeOpTorch.

State

The state is the value calculated by the public method ⁠$shapes_out()⁠.

Credit

Part of this documentation have been copied or adapted from the documentation of torch.

Parameters

• eps :: numeric(1)
  A value added to the denominator for numerical stability. Default: 1e-5.

• momentum :: numeric(1)
  The value used for the running_mean and running_var computation. Can be set to NULL for cumulative moving average (i.e. simple average). Default: 0.1

• affine :: logical(1)
  a boolean value that when set to TRUE, this module has learnable affine parameters. Default: TRUE

• track_running_stats :: logical(1)
  a boolean value that when set to TRUE, this module tracks the running mean and variance, and when set to FALSE, this module does not track such statistics and always uses batch statistics in both training and eval modes. Default: TRUE

Internals

Calls torch::nn_batch_norm1d(). The parameter num_features is inferred as the second dimension of the input shape.

Super classes

mlr3pipelines::PipeOp -> mlr3torch::PipeOpTorch -> mlr3torch::PipeOpTorchBatchNorm -> PipeOpTorchBatchNorm1D

Methods

Public methods

• PipeOpTorchBatchNorm1D$new()

• PipeOpTorchBatchNorm1D$clone()

Method new()

Creates a new instance of this R6 class.

Usage

PipeOpTorchBatchNorm1D$new(id = "nn_batch_norm1d", param_vals = list())

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

PipeOpTorchBatchNorm1D$clone(deep = FALSE)

Arguments

See Also

Other PipeOps: mlr_pipeops_nn_avg_pool1d, mlr_pipeops_nn_avg_pool2d, mlr_pipeops_nn_avg_pool3d, mlr_pipeops_nn_batch_norm2d, mlr_pipeops_nn_batch_norm3d, mlr_pipeops_nn_block, mlr_pipeops_nn_celu, mlr_pipeops_nn_conv1d, mlr_pipeops_nn_conv2d, mlr_pipeops_nn_conv3d, mlr_pipeops_nn_conv_transpose1d, mlr_pipeops_nn_conv_transpose2d, mlr_pipeops_nn_conv_transpose3d, mlr_pipeops_nn_dropout, mlr_pipeops_nn_elu, mlr_pipeops_nn_flatten, mlr_pipeops_nn_gelu, mlr_pipeops_nn_glu, mlr_pipeops_nn_hardshrink, mlr_pipeops_nn_hardsigmoid, mlr_pipeops_nn_hardtanh, mlr_pipeops_nn_head, mlr_pipeops_nn_layer_norm, mlr_pipeops_nn_leaky_relu, mlr_pipeops_nn_linear, mlr_pipeops_nn_log_sigmoid, mlr_pipeops_nn_max_pool1d, mlr_pipeops_nn_max_pool2d, mlr_pipeops_nn_max_pool3d, mlr_pipeops_nn_merge, mlr_pipeops_nn_merge_cat, mlr_pipeops_nn_merge_prod, mlr_pipeops_nn_merge_sum, mlr_pipeops_nn_prelu, mlr_pipeops_nn_relu, mlr_pipeops_nn_relu6, mlr_pipeops_nn_reshape, mlr_pipeops_nn_rrelu, mlr_pipeops_nn_selu, mlr_pipeops_nn_sigmoid, mlr_pipeops_nn_softmax, mlr_pipeops_nn_softplus, mlr_pipeops_nn_softshrink, mlr_pipeops_nn_softsign, mlr_pipeops_nn_squeeze, mlr_pipeops_nn_tanh, mlr_pipeops_nn_tanhshrink, mlr_pipeops_nn_threshold, mlr_pipeops_nn_unsqueeze, mlr_pipeops_torch_ingress, mlr_pipeops_torch_ingress_categ, mlr_pipeops_torch_ingress_ltnsr, mlr_pipeops_torch_ingress_num, mlr_pipeops_torch_loss, mlr_pipeops_torch_model, mlr_pipeops_torch_model_classif, mlr_pipeops_torch_model_regr

Examples

# Construct the PipeOp
pipeop = po("nn_batch_norm1d")
pipeop
# The available parameters
pipeop$param_set

Description

Applies Batch Normalization for each channel across a batch of data.

Input and Output Channels

One input channel called "input" and one output channel called "output". For an explanation see PipeOpTorch.

State

The state is the value calculated by the public method ⁠$shapes_out()⁠.

Credit

Part of this documentation have been copied or adapted from the documentation of torch.

Internals

Calls torch::nn_batch_norm2d(). The parameter num_features is inferred as the second dimension of the input shape.

Parameters

• eps :: numeric(1)
  A value added to the denominator for numerical stability. Default: 1e-5.

• momentum :: numeric(1)
  The value used for the running_mean and running_var computation. Can be set to NULL for cumulative moving average (i.e. simple average). Default: 0.1

• affine :: logical(1)
  a boolean value that when set to TRUE, this module has learnable affine parameters. Default: TRUE

• track_running_stats :: logical(1)
  a boolean value that when set to TRUE, this module tracks the running mean and variance, and when set to FALSE, this module does not track such statistics and always uses batch statistics in both training and eval modes. Default: TRUE

Super classes

mlr3pipelines::PipeOp -> mlr3torch::PipeOpTorch -> mlr3torch::PipeOpTorchBatchNorm -> PipeOpTorchBatchNorm2D

Methods

Public methods

• PipeOpTorchBatchNorm2D$new()

• PipeOpTorchBatchNorm2D$clone()

Method new()

Creates a new instance of this R6 class.

Usage

PipeOpTorchBatchNorm2D$new(id = "nn_batch_norm2d", param_vals = list())

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

PipeOpTorchBatchNorm2D$clone(deep = FALSE)

Arguments

See Also

Other PipeOps: mlr_pipeops_nn_avg_pool1d, mlr_pipeops_nn_avg_pool2d, mlr_pipeops_nn_avg_pool3d, mlr_pipeops_nn_batch_norm1d, mlr_pipeops_nn_batch_norm3d, mlr_pipeops_nn_block, mlr_pipeops_nn_celu, mlr_pipeops_nn_conv1d, mlr_pipeops_nn_conv2d, mlr_pipeops_nn_conv3d, mlr_pipeops_nn_conv_transpose1d, mlr_pipeops_nn_conv_transpose2d, mlr_pipeops_nn_conv_transpose3d, mlr_pipeops_nn_dropout, mlr_pipeops_nn_elu, mlr_pipeops_nn_flatten, mlr_pipeops_nn_gelu, mlr_pipeops_nn_glu, mlr_pipeops_nn_hardshrink, mlr_pipeops_nn_hardsigmoid, mlr_pipeops_nn_hardtanh, mlr_pipeops_nn_head, mlr_pipeops_nn_layer_norm, mlr_pipeops_nn_leaky_relu, mlr_pipeops_nn_linear, mlr_pipeops_nn_log_sigmoid, mlr_pipeops_nn_max_pool1d, mlr_pipeops_nn_max_pool2d, mlr_pipeops_nn_max_pool3d, mlr_pipeops_nn_merge, mlr_pipeops_nn_merge_cat, mlr_pipeops_nn_merge_prod, mlr_pipeops_nn_merge_sum, mlr_pipeops_nn_prelu, mlr_pipeops_nn_relu, mlr_pipeops_nn_relu6, mlr_pipeops_nn_reshape, mlr_pipeops_nn_rrelu, mlr_pipeops_nn_selu, mlr_pipeops_nn_sigmoid, mlr_pipeops_nn_softmax, mlr_pipeops_nn_softplus, mlr_pipeops_nn_softshrink, mlr_pipeops_nn_softsign, mlr_pipeops_nn_squeeze, mlr_pipeops_nn_tanh, mlr_pipeops_nn_tanhshrink, mlr_pipeops_nn_threshold, mlr_pipeops_nn_unsqueeze, mlr_pipeops_torch_ingress, mlr_pipeops_torch_ingress_categ, mlr_pipeops_torch_ingress_ltnsr, mlr_pipeops_torch_ingress_num, mlr_pipeops_torch_loss, mlr_pipeops_torch_model, mlr_pipeops_torch_model_classif, mlr_pipeops_torch_model_regr