can't call as_array() on a torch_float32() tensor on "mps" (M1 Mac Studio) GPU

[masp]

I've been trying to get some torch code working on an M1 Mac Studio (device = "mps"): I had to specify dtype = torch_float32() to get it to run, but it works (the results are strange but I haven't looked into that yet): however, when I try to call as_array() I get an error that seems to indicate it's trying to convert the tensors back to regular R types via a torch_float64(): here's a reproducible example:

> x <- torch_tensor(1, device = "mps", dtype = torch_float32())
> x
torch_tensor
 1
[ MPSFloatType{1} ]
> as_array(x)
Error in cpp_as_array(x$ptr) :
  Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.
Exception raised from empty_mps at /Users/dfalbel/Documents/actions-runner/mlverse/_work/libtorch-mac-m1/libtorch-mac-m1/pytorch/aten/src/ATen/mps/EmptyTensor.cpp:40 (most recent call first):
frame #0: at::detail::empty_mps(c10::ArrayRef<long long>, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, c10::optional<c10::MemoryFormat>) + 1048 (0x16f464f0c in libtorch_cpu.dylib)
frame #1: at::native::empty_strided_mps(c10::ArrayRef<long long>, c10::ArrayRef<long long>, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>) + 140 (0x16f46d278 in libtorch_cpu.dylib)
frame #2: at::_ops::empty_strided::redispatch(c10::DispatchKeySet, c10::ArrayRef<long long>, c10::ArrayRef<long long>, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<

Is there a workaround? NB: I filed an issue (can't call as_array() on a torch_float32() tensor on "mps" (M1 Mac Studio) GPU · Issue #972 · mlverse/torch · GitHub) but it hasn't been picked up and I guess may not be, so maybe this is a more appropriate forum for seeking help?

thanks!

[technocrat]

@masp

framework doesn't support float64. Please use float32 instead.

Looks like it ran into a 64 bit object in the process of running, either from data or return value in R function that returns one. Think you need to fix the initial install issue.

[andresrcs]

Cross-posted here:

can't call as_array() on a torch_float32() tensor on "mps" (M1 Mac Studio) GPU

Please familiarize yourself with our crossposting policy

FAQ: Is it OK if I cross-post?

Posting the same question both here and on other sites Posting the same question to multiple forums at the same time is often considered impolite. We don't completely ban such cross-posting, but we ask you to think hard before you do it and to follow some rules. Cross-post sparingly Rather than post the same thing here and elsewhere from the get-go, post in one place at a time. Let enough time go by (think days, not hours) before you take your question somewhere else. Sometimes people at another site may suggest you post here if your question doesn't fit within the scope of the other site. Always link to your other posts, and update everywhere with any solution…

[masp]

@technocrat Thanks! What's the initial install issue?

@andresrcs Sorry about the cross-posting: I did it because my question in the github issue tracker has been unanswered, so I'm assuming I should not have asked this question in that forum.

[andresrcs]

Cross-posting is not necessarily a bad thing, just make sure to follow our guidelines when you do it here.

[masp]

Thanks: I just realised I'd neglected to paste in the URL to the issue in my original post; edited to fix just now. (Also: it's been 5 days since my original issue was created, hence I'm doubting it will get an answer.)

[technocrat]

@masp

had to specify dtype = torch_float32() to get it to run

may be the root cause

[masp]

Interesting: I specified torch_float32() because of the following:

> x <- torch_tensor(1, device = "mps")
> x %>% as_array
Error in cpp_as_array(x$ptr) :
  Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.
Exception raised from empty_mps at /Users/dfalbel/Documents/actions-runner/mlverse/_work/libtorch-mac-m1/libtorch-mac-m1/pytorch/aten/src/ATen/mps/EmptyTensor.cpp:40 (most recent call first):
frame #0: at::detail::empty_mps(c10::ArrayRef<long long>, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, c10::optional<c10::MemoryFormat>) + 1048 (0x14ba28f0c in libtorch_cpu.dylib)
frame #1: at::native::empty_strided_mps(c10::ArrayRef<long long>, c10::ArrayRef<long long>, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>) + 140 (0x14ba31278 in libtorch_cpu.dylib)
frame #2: at::_ops::empty_strided::redispatch(c10::DispatchKeySet, c10::ArrayRef<long long>, c10::ArrayRef<long long>, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<

But even with it I get the same error on converting back to non-tensor types:

> y <- torch_tensor(1, device = "mps", dtype = torch_float32())
> y %>% as_array
Error in cpp_as_array(x$ptr) :
  Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.
Exception raised from empty_mps at /Users/dfalbel/Documents/actions-runner/mlverse/_work/libtorch-mac-m1/libtorch-mac-m1/pytorch/aten/src/ATen/mps/EmptyTensor.cpp:40 (most recent call first):
frame #0: at::detail::empty_mps(c10::ArrayRef<long long>, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>, c10::optional<c10::MemoryFormat>) + 1048 (0x14ba28f0c in libtorch_cpu.dylib)
frame #1: at::native::empty_strided_mps(c10::ArrayRef<long long>, c10::ArrayRef<long long>, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<c10::Device>, c10::optional<bool>) + 140 (0x14ba31278 in libtorch_cpu.dylib)
frame #2: at::_ops::empty_strided::redispatch(c10::DispatchKeySet, c10::ArrayRef<long long>, c10::ArrayRef<long long>, c10::optional<c10::ScalarType>, c10::optional<c10::Layout>, c10::optional<

Is there another type I should specify?

[technocrat]

I’ll see what I can figure out about the MPS thingie. Ping me after a decent interval?

[masp]

Many thanks! I hope you can help me: you're my only hope

[technocrat]

I’m only as old as Obi-Wan. The error is ultimately due to the fact that neither does the Apple GPU, nor its mps framework, handle the double-precision float64 data type. Try setting device to cpu, and if it doesn’t throw the same error, that’s probably what’s happening.

Not sure how to overcome the obstacle without learning more about the overall organization of the workflow. Report back?

[masp]

(I'm glad you caught my reference!)

Just saw a reply to my github issue: can't call as_array() on a torch_float32() tensor on "mps" (M1 Mac Studio) GPU · Issue #972 · mlverse/torch · GitHub

I tested it and it works:

> y <- torch_tensor(1, device = "mps", dtype = torch_float32())
> as_array(y$to(device = "cpu"))
[1] 1

Interestingly enough, so does this:

> x <- torch_tensor(1, device = "mps")
> as_array(x$to(device = "cpu"))
[1] 1