Snape's love for Lily was the main factor behind his drive to protect her only son, for whom she had sacrificed her life. The bitter irony for Snape was that this same boy was a living reminder of the fact that, once again, James Potter had trumped him; in this case, in a much more personal way than being better at Quidditch: James won over Lily.
So Snape had nothing but resentment for Harry from their first meeting simply because he was James Potter's son - and the fact that Harry bore so much of a physical resemblance to his father did not help, neither did Harry's apparently inherited Quidditch skills. Snape was determined to loathe him, despite Harry's virtuous personality traits.
Snape: "- mediocre, arrogant as his father, a determined rule-breaker, delighted to find himself famous, attention-seeking and impertinent -"
Dumbledore: "You see what you expect to see, Severus ... Other teachers report that the boy is modest, likeable and reasonably talented. Personally, I find him an engaging child." --- Harry Potter and the Deathly Hallows, Chp.33 The Prince's Tale.
However, when the time came in Deathly Hallows, Snape is shown to have understood enough about Harry to know that he would follow his doe Patronus into the dark woods of the Forest of Dean. Harry senses nothing sinister in Snape's Patronus, seeing as the doe is a reflection of Lily Potter - the bridge between these two characters. Which brings me to my next point.
Now, this is mostly my speculation, but I'm almost certain that I'm right when I say that Snape probably felt some bond or relation to Harry very, very deep down. It wasn't just a favor to Lily, there was some recognition of Harry as a kindred spirit there as well; why else would Snape use his doe Patronus to lure Harry to Gryffindor's Sword? He must have understood Harry's desire to be with his mother, as well as Dumbledore's words to him.
Snape: "He [Harry] is his father over again -"
Dumbledore: "In looks, perhaps, but his deepest nature is much more like his mother's." --- Harry Potter and the Deathly Hallows, Chp. 33 The Prince's Tale.
I believe that Snape must have seen Lily's personality in Harry at some point, not just his father's. I think that saying Snape's care for Harry extended only to the level of honoring Lily's sacrifice just scratches the surface. True, Snape denied that he cared for Harry at all to Dumbledore ("For him?), but I just think his pride prevented him from admitting it. Snape's choice of words, in his indignation at Dumbledore's plan to have Harry sacrifice himself to Voldemort, is very suggestive of some concern for Harry himself.
Snape: Now you tell me you have been raising him [Harry] like a pig for slaughter -" --- Harry Potter and the Deathly Hallows, Chp. 33 The Prince's Tale.
Like I said above, Snape brushes Dumbledore's question as to whether he is concerned for Harry aside by casting the silver doe, but I think this is partly a denial. Just my speculation! :)
Overall, I really think Snape, despite all his resentment, anger and bitterness towards Harry over his years at Hogwarts, was not completely indifferent to Harry's suffering. I think there was a part of him that genuinely cared for Harry as much as he did for his mother Lily. There was more to Snape and Harry's relationship than Snape trying to ensure Harry's survival.