Potential employer contacting co-workers w/o permission

I am in the process of looking for a new job. I plan on working for a contractor under a large services contract.

The program manager in charge of the contract has been working with the company I am going to work for to check my qualifications for the position. But last week the program manager (who is NOT an employee of the company I am going to work for) personally knows some people at my current office. He called them to check up on me. I am not worried about what they said to him about me. Instead, my concern is that those people he spoke with went to the managers in my office and told them directly that I must be looking for a new job Regardless of what I say people will be convinced I am leaving shortly. When people in this office are leaving the attitude toward them is not pleasant, and I am already beginning to experience some of that.

In conclusion: I did not give out references, the guy called people he had worked with here for certain projects. I do not have a solid job offer yet. The person that did this does not even work for the company that is hiring me. If his actions cause problems for me, is there any recourse?

Warn him to stop.

If he does not he could be liable legally. But almost certainly if you say something it will not happen again/