Initially I judge by the story. Once I get farther in or get the interest to draw from it, I'll start looking at the graphics and design more. But some anime can be awesome without much design effort.
Take Soul Eater for example. Everything is so simple. ( And I swear that the sky is superimposed. It pixelates off of stuff. Woah.) And yet it has such an awesome and unique story, and became really popular really fast.
But take another anime like Tegami Bachi. Tegami Bachi is beautifully designed, and the storyline is gorgeous as well; however, it isn't as popular. This probably has to do with how quickly it can put out. Given that it takes longer to make the anime more appealing to the eye, it can't put out as many episodes in a given amount of time. Therefor, people tend to lose interest.
I suppose it comes down to a medium. Graphics play a role in getting attention and keeping up the pace, whereas the plot holds us into it and lets us enjoy the ride.

^
Dang, that's a lot of text.

I'm a B-Anime Addict and C!

guess that makes me a B Addict too

Hmm, I say I'm a B- Anime Addict. I need both a good story and at least a so-so graphic to get me interested.