Do votes from instances that aren’t federated with your instance still affect the score on posts you do federate with?

For example, if im on an instance that only federates with instances in an allowlist can someone create an instance of bots to downvote things in the lemmy.ml instance and affect the score of the lemmy.ml posts that users on my instance can see?

I’m interested in learning how lemmy prevents vote brigading from instances that might be made with malicious intent