Odin raised both Thor and Loki as princes of Asgard ready for the throne but knew only one of them would get that chance, it happened to be Thor.
Odin: Only one of you can ascend to the throne. But both of you, were born to be kings.
In Loki’s eyes Thor probably lost the right to the throne forever after Odin cast him out in Thor. In that case the right to the throne would have passed to him and hence why it was not his birthright: with Thor having lost that right.
Of course he also still feels like as the son of Laufey he is owed a throne as we see come out later in life when talking to Thanos.
Loki: I, Loki, Prince of Asgard, Odinson, the rightful King of Jotunheim, God of Mischief, do hereby pledge to you, my undying fidelity