I think result1 and result2 are supposed to be the same and yet R gives very different result. Could someone explain why there is a difference and how to replicate the result from glmnet with the tidymodel way?
This has to do with how we fit the glmnet model in tidymodels. Even when you give a penalty, we let it compete the full path (see the backstory here). We do say in the help file that:
For glmnet models, the full regularization path is always fit regardless of the value given to penalty .
This changes the lambda path in the objects:
> result1$lambda
[1] 0
> str(result2$fit$lambda)
num [1:100] 4060 3699 3370 3071 2798 ...
> summary(result2$fit$lambda)
Min. 1st Qu. Median Mean 3rd Qu. Max.
0.406 4.063 40.640 456.932 406.297 4059.624
For this type of model, when a penalty value that is not included in the path is used, it does linear interpolation between the values. Since the path values do not bracket your value (zero), it does a poor job at interpolation. I don't think that they expected people to fit an unregularized model with this function.
You can get closer by expanding the range of penalty values using lambda.min.ratio: