I think the problem with the show is that is based on a comic series, and they don't follow it to the letter, and when they drive away from the story in the comics the comic fans get really upset. I have tried reading a bit of the comics and while it was alright it didn't got me addicted as much as the show, perhaps im not much of a comics guy.
I do believe season 3 was better compared to the second one, they heard some of the viewers suggestions to improve. However with the season 3 finale i still have my doubts about the season 4, the finale was a bit slow paced and i don't see much to look for for the next season. (Don't wanna spoil anyone so not saying any particular reasons.)