I am not with my childs father. I am trying to leave the country, he says I can't. Is that true?

I was never married to my childs father. I have not been with him in 2 years. I am now trying to go work in Japan and take my 3 year old daughter (who lives with me). Her father says I cant do that because he doesnt want her to leave. Can he stop me? If We are not married dont I have full custody of the child unless proven otherwise? He just recently(in January) started paying a child support ( a measly $189.00) because he doesnt have a job. Does that give him any right? His name is also on the birth certificate. I also tried to explain to him that I would get like 10 weeks of vacation and I would bring her back to visit, but he is trying to give me a hard time. This is the opportunity of my life , I have a BFA Degree and I cannot find a decent job here. He has nothing (no degree, no house, no job) and he is 11 years older than me and trying to stop me from doing whats best for me and my child. Some one help!! What can I do??