Accessing low-level API from other packages

[Enchufa2]

Developing a package that plugs into torch is a big problem that you solved via lantern, so it would be nice to export that solution for other packages. In fact, you partially did, putting the API in inst/include, but it's not currently usable.

Basically I have an algorithm in C++, so going through the R API is not an option, and it would be great to have access to torch directly, same access as e.g. RcppEigen or RcppArmadillo provide. In other words, it would be nice to be able to do something like

#include <Rcpp.h>
using namespace Rcpp;

// [[Rcp::depends(torch)]]
#include <torch.h>

// [[Rcpp::export]]
torch::Tensor zeros(int rows, int cols) {
  int64_t sizes[2] = {rows, cols};
  torch::IntArrayRef sizes_ref = torch::IntArrayRef(sizes);
  torch::TensorOptions options = lantern_TensorOptions();
  torch::Dtype dtype = lantern_Dtype_float64();
  options = lantern_TensorOptions_dtype(options.get(), dtype.get());
  return lantern_zeros_intarrayref_tensoroptions(sizes_ref.get(), options.get());
}

/*** R
zeros(5, 3)
*/

If this worked, we could basically embed access to torch in C++ code without routing anything through R. AFAICT, to provide this, torch would need to:

• Provide the mechanism to inject the includes into CPPFLAGS.
• Provide a low-level API point to start lantern.
• Register a bunch of functions like deleters and so on, so that objects in the torch namespace work.

[dfalbel]

Hi @Enchufa2

We have some sort of support for this. See here: https://github.com/mlverse/lltm for a tutorial and https://github.com/mlverse/torchvisionlib for a more complete example.

[Enchufa2]

Nice, didn't know this, thanks for the pointer. If I understand correctly, this allows third-party packages to use torch's C++ API directly, but then they need to build their own binaries similar to what this package does, right? Because a libtorch bundle is required for building. What I'm proposing here would be to use lantern directly, which would not require building separate binaries if I'm not mistaken.

[dfalbel]

Ohh I see. Yeah, this may be possible, but lantern has a C interface which is really not convenient to use directly. FWIW for simple algorithms like what you have here, you can just copy the declarations of lantern_ from lantern.h.

torch::Tensor zeros(int rows, int cols) {
  int64_t sizes[2] = {rows, cols};
  torch::IntArrayRef sizes_ref = torch::IntArrayRef(sizes);
  torch::TensorOptions options = lantern_TensorOptions();
  torch::Dtype dtype = lantern_Dtype_float64();
  options = lantern_TensorOptions_dtype(options.get(), dtype.get());
  return lantern_zeros_intarrayref_tensoroptions(sizes_ref.get(), options.get());
}

In an ideal world, we could have torch.h exporting the full torch C++ api, rebuilt from just from lantern_ operations.
In LLM times this maybe possible to build :)

[Enchufa2]

Ohh I see. Yeah, this may be possible, but lantern has a C interface which is really not convenient to use directly. FWIW for simple algorithms like what you have here, you can just copy the declarations of lantern_ from lantern.h.

Yes, we have a relatively simple set of operations that we currently solve with RcppArmadillo here (note: the linalg_torch.h file there is just for local tests directly over libtorch), and we wanted to try with torch. So I thought that even if lantern is cumbersome, it would relatively easy to port. But the approach in torchvisionlib is even more cumbersome for this case.

In an ideal world, we could have torch.h exporting the full torch C++ api, rebuilt from just from lantern_ operations. In LLM times this maybe possible to build :)

That would be really cool. :)

[dfalbel]

@Enchufa2

Maybe you want to take a quick look at: https://github.com/mlverse/torch/pull/1470/changes#diff-8e93f436a870234728e18c8d63792a2a58f2ee6edda9b4e7295ac8b893fa5185

to see if it matches what you are thinking.

[Enchufa2]

Wow, that was fast. :) It looks very good. Things I'm missing after a quick look:

• Method cat, although uses of this one can (and maybe should) be replaced by something else.
• It is not clear how to deal with XPtrTorchoptional_int64_t. It seems that it is not possible to just pass an int, i.e. that the compiler doesn't know a conversion?
• Operators like operator* with scalars.
• How to get the options from a given tensor to copy them to another. Or is this handled automatically?