Visitation. Child lives in different state than father..

I am hoping that someone could help me with the question I have about visitation. I was married to my ex for 6 yrs I had a daughter before we married and one with him. My children are now 11 and 9. We were divorced in 2004 in the state of IL and at that time the judge ruled that he pay cs for both children even though my oldest is not his child. Well right after the divorce we (the kids and I) move to Washington state with permission from the courts. We did not set up any visitation plans though. He has not seen the kids at all since 2004. He calls once in awhile and sends gifts on holidays and birthdays. He will once in awhile request to see them but never follows though. He is also remarried with 2 children. Now here is the problem. He is wanting the kids to go to IL from WA. And we all have concerns (my family) that we would never get the youngest home again because he has already threatened to take her. But in YEARS he has wanted nothing to with her even if we did live with him (we lived apart alot when we were married)
So we are worried if we sent them there we would never get the youngest one back.. we would have no problem with the oldest because of not being his child. So my question is.. What rights do we have rights? Does he have to come to the state the children reside in? Is it possible to make it so that he does have to come to the state the children reside in for visitation? Please help. It is not that I do not want him to see the kids but I do not want to have to worry I will never see them again..
Thanks very much and any advice would be wonderful

I'm not a lawyer, but i know what your going through and how worried you must be. My opinion is call a local lawyer and ask them because honestly they'd be the ones that "should" know the laws that would pertain to you and your worries. As far as does he have to come to the state that the children reside in ... i dont think he does, but i think at the least he'd be the one that has to pay for the plane tickets or whatever, atleast one way, there are times that the courts feel that the parents each must pay for the trips so one would pay for the tickets to get the children to where their going to visit and then the other would pay for the tickets to get the children back home, this i believe might vary depending on all. Don't get too worried tho, remember u urself stated that the father has only ever made comments about the children goin to visit him, but that he has never followed through. You might just luck out without doing anything, but then too if he pushes it once he finds out how much the tickets and all will cost he might really back off. To be on the safe side i'd suggest the first thing u do is ask a local lawyer.

Good Luck!