It’s OK to delete comments that are off-topic, mean, or just unpleasant. It’s your blog, and what’s said there reflects on you.
Don’t let a drive-by comment forever diminish the value of your work. Your job is to take care of all of your readers and give them a great experience, not to provide free publishing to one angry person.