We don't really have enough info to help you out. Could you ask this with a minimal REPRoducible EXample (reprex)? A reprex makes it much easier for others to understand your issue and figure out how to help.
Maybe a walk around solution would be to use a boolean variable (TRUE/FALSE = 1/0) instead of a factor that way you can use rowsum directly.