.A threatens to call the cops if I take my son out of the house without his permission
The behavior that you describe of the other parent threatening to call the police if you take the child out when s/he says ‘no’ is worrisome and sounds like it could be a symptom of abuse. Do you feel that he often tries to control you and your actions. Does he try to control other aspects of your life such as what you wear, where you go, how much money you spend, etc? If so, this type of control, especially if it is coupled with emotionally or physically or sexually abusive behavior may be considered part of domestic violence. Here is a link that has a checklist of abusive behaviors so so you can see if you may be in an abusive relationship: https://www.womenslaw.org/about-abuse/am-i-being-abused/signs-abuse and here is more info on domestic violence: https://www.womenslaw.org/domestic-violence
Now, to address the issue of the other parent’s threats to call the cops, I am not specifically familiar with [STATE] laws or how the police would handle this. You may want to ask an attorney in your county or state if the police might respond to such a call or not and if they respond, whether or not they may get involved with who the baby can go with. You may want to ask an attorney if the police may say that you are both the child’s parents and have equal control over what the child does or where he goes and that either can decide to take the child someplace. Again, you may want to call a local legal organization to ask for advice on this. You may want to start out by calling: [STATE] Coalition Against Domestic Violence - [GIVE CONTACT INFO] and if they can’t help you, you may want to call a legal services organization and ask to have a consultation with a lawyer who specializes in custody and/or domestic violence issues - here is a link for free lawyers:LINK