Hello Samuel,
I am afraid I can only help you by 'touching' your post, so that it is seen (again) by community members that know your environment: "apple silicon" and "xcode" are not known to me.
On the other site: it is not unusual for the install of a package that a lot of dependencies are also installed.
I don't know if in your environment packages have to be compiled.
In the Windows environment where I work, you can sometime choose if you want to install the binary or the source version, the latter having to be compiled.
I hope your problem is being solved soon!