Insurance Agent in Missouri

Missouri - I have worked for an agency for 6 years and quit 3 weeks ago to work for another agency. My former employer was okay with that because it was 26 miles away from home. I was offered a better offer back in my hometown with one of my former employer's old business partners and now he is threatning to sue me if I solicit any business that is his. We were made to sign a customer privacy policy in 2005 and he has sent me a copy of this form. We were told we would sign or loose our job so everyone signed it. The following is the one paragraph that would pertain to
the solicitation of his business. Could you please read this and tell me if he has any chance of preventing me from working at this new agency?

"In addition, employees have access during their employment to confidential information of the Agency not available to the public, such as Agency customer and contact lists, prospects and projections, policy and expiration information and information regarding customer operations. This information is protected by law as trade secrets of theAgency as its sole and exclusive property. Employees shall not, whether during or after their employment with the Agency, directly or indirectly allow the use of such protected information by any other party. This would include using Agency customer information for the purpose of soliciting customers for insurance coverage through any future employer."

Now I do not have any llists from his agency but I had customers that I am sure will come to me when they find out I am back in town.

Thanks for your advise.