Neighbors want to sue over fence

[gumbi17]

About 4 months ago we had a storn that knocked down one peice of a fence that separated my neighbors fence and mine. This was the orginal fence that had been put up when all the houses were built. Well one morning my wife got a phone call that they were tearing down the fence and building a new one, she looked out the window and people were already starting to work. Well it took 3 days and te fence went up and they stained their part of the fence, they never stained the part that faced ours. When they stained the fence all of the knots in the wood bled thru, so it looked horible on my side of the fence. Well after 3 months we were planning a party for my daughter getting married and started staining the fence on our side. Well after the fence was stained they came over and said they wanted it off and they wanted us to have plain wood on our side again. They never planned to have anything but their color on it. Now they just sent us a letter saying we owe $4700 to redue the fence or they will take us to court. They never got a building permit to have their entire fence rebuilt. Will the court actually take this case or what is our recourse? We also live in Texas.

Whose property is the fence on and what was the agreement for maintenance?