Yes

Harry's parents are already dead. They know that there is not much to fear from death. Thus they have much less reason than living parents not to want Harry to die. That said, they understand the impact that death has on those in the world of the living, and therefore wish for there to be an end to Voldemort's reign of terror. Being dead, they no longer fear death, but there are still other reasons to kill Voldemort. As Dumbledore (who is dead) says:

“Do not pity the dead, Harry. Pity the living, and above all, those who live without love. By returning, you may ensure that fewer souls are maimed, fewer families are torn apart.

—Harry Potter and the Deathly Hallows

That, presumably, is what Lily and James are proud of Harry for preventing.

The lines quoted in your question make their attitude clear:

“Very close. We are . . . so proud of you.”

—Harry Potter and the Deathly Hallows

They are proud of him because he is willing to sacrifice his life, which, even knowing of the existence of an afterlife, means giving up all his friends for an unknown fate. Don't forget that both Lily and James in some sense sacrificed themselves so that Harry could live, making it possible that they would have been proud of him even in life.Don't forget that both Lily and James in some sense sacrificed themselves so that Harry could live, making it possible that they would have been proud of him even in life. In addition, given that they were dead, they might have looked forward to seeing Harry again.

Dumbledore's spirit had a similar attitude:

“Harry,” He spread his arms wide, and his hands were both whole and white and undamaged. “You wonderful boy. You brave, brave man. Let us walk.”

—Harry Potter and the Deathly Hallows