Is this the right place for tensorflow bugs?
Predictions may differ slightly depending on position and input length. More specifically, if n = NROW(x)
, row i
will get one value if 0 < i <= bitwAnd(n, -4)
, but a possibly different value for bitwAnd(n, -4) < i <= n
. (These are FORTRAN style 1-based indexes. For C++ / Python 0-based, switch "<=" and "<".) I write "may" as it happens to me with probability around 0.4. "Slightly" means in the order of the least significant bits of the float32 mantissa.
Here's a reproducible example:
options(digits=8)
fake <- function(shape_) { # arbitrary but reproducible
array(seq_len(prod(shape_)) %% 2.71 - 1.04, shape_)
}
library(keras)
shape <- c(30,5)
model <- keras_model_sequential() %>%
layer_lstm(units=2, input_shape=shape) %>%
set_weights(list(fake(c(5, 8)), fake(c(2, 8)), fake(8)))
n <- 11 # not a multiple of 4
x <- array(rep(fake(shape), each=n), c(n, shape)) # n copies of identical input
p <- model %>% predict(x) # all predictions should match
p # but last n%%4 rows differ
#> [,1] [,2]
#> [1,] 0.46561426 -0.22865930
#> [2,] 0.46561426 -0.22865930
#> [3,] 0.46561426 -0.22865930
#> [4,] 0.46561426 -0.22865930
#> [5,] 0.46561426 -0.22865930
#> [6,] 0.46561426 -0.22865930
#> [7,] 0.46561426 -0.22865930
#> [8,] 0.46561426 -0.22865930
#> [9,] 0.46561423 -0.22865926
#> [10,] 0.46561423 -0.22865926
#> [11,] 0.46561423 -0.22865926
(t(p)-p[1,]) * 2**26 # the difference is low bits
#> [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11]
#> [1,] 0 0 0 0 0 0 0 0 -2 -2 -2
#> [2,] 0 0 0 0 0 0 0 0 3 3 3
Created on 2019-07-22 by the reprex package (v0.3.0)
One odd consequence is that may get one result in row 25 when there were 28 rows total but a different result when there are 27 total (so 25 was in the last 27%%4 = 3), without otherwise changing the input data.