Can he keep my security deposit?

I need a little advice...

I rented an apartment on Jan 12. I signed a month-to-month agreement.
Three days later, I circumstances prevented me from being able to move in.
I told the landlord I had to break the agreement. I never moved into the apartment. We've straightened out the pro-rated rent owed situation, but now he's saying he's keeping part of my security deposit because he says "legally" he can charge me $100.00 for repainting the apartment.

Can he?

Well not normally for re-painting but he can charge for having to advertise again and find new people etc.