To Aunt Petunia.
The protection specifically applies to the house where Petunia lives. If Petunia moves house then so does the protection.
“While you can still call home the place where your mother’s blood dwells, there you cannot be touched or harmed by Voldemort. He shed her blood, but it lives on in you and her sister. Her blood became your refuge. You need return there only once a year, but as long as you can still call it home, there he cannot hurt you."
(Order of the Phoenix, Chapter 37, The Lost Prophecy).
Obviously, the protection applies to the house and not to Petunia herself; Harry doesn't have to be physically nearby Petunia to get the protection. She can leave the house for the evening and Harry is still protected. What is important is that Petunia "dwells" there, that both she and Harry can call that place home.