Daughter is with me upon father's request

My ex and I have joint custody of our 9 yr old daughter. Joint custody was mutualy established in Long Island NY's about 5 years ago. Her father received residential custody at the time, since I did not yet have my own place. Since then, our daughter has shifted back and fourth between us, staying a year with her father and a year with me, mostly do to her father requests. I would like her to remain with me, as I raised her since she was a baby even when we split up. Whenever she ends up staying with her father, he eventully calls to have me pick her up and live with me. The first time he did this, I had him write a statement saying he is letting her live with me. He drives a truck now, he has been through two girlfriends, one of which physicaly abused her, another of which verbally abuses and exploits her.

My daughter means the world to me and I do not want keep seeing her go back and forth between her father and I like this. I have since moved to the same state where her father lives, so that she can be near both of us. I have my own place, am attending college, have gotten her registered into school. She has health coverage of which her father never got her. I am arranging for her to be in therapy, because everytime she comes back to me after being with her father, she will have outbursts. I believe this is because of the instability and mental intimidation his girlfriends put her through. She is making friends here and is adjusting well. Now if her father just shows up and demands that he take her back, what rights do I have? Technicaly from 5 years ago he still has "residential custody". At least in NY. I am on SSI and have been diagnosed with depression in the past. He has threatened to use this against me in court. What should I do to protect the best interest of my child in this situation? I am tired of being afraid of her father and his "girlfriends".

If your recent health etc. has been fine it is not likely a court will hold that against you. You could also invite counselors or others to be a part of you care--if the court has any concerns.