Why is my cat so aggressive toward specific people?

Your father probably carries the scent of his 4 cats and your tomcat reacts to that. Cats will act the same way if you carry the scent of a dog. Tell your dad to take a shower. That should suffice.