Quite a lot of details match with Foreigner by C.J. Cherryh (as per the comment by DannyMcG). Quite a lot of detail differ.

• Humans are stranded on an alien world. The main part of the story is several centuries after the landing, and the main character (Bren Cameron) is a trained diplomat, not a pilot. There is an introduction about how the humans got shipwrecked in the first place, which features bridge crew.
• The aliens are less advanced than the humams, but catching up quickly due to a land-for-tech deal. No longer nomadic.
• Outside the major towns, riding animals (mecheiti) are used which match your description.
• Cameron is taken captive by a faction of the aliens.
• Cameron is poisoned by alien tea. An alien physician, without access to human-specific drugs, prescribes a glass of milk every three hours until the milk no longer tastes bitter like the tea. They serve warm milk, but a half-dazed Cameron insists on cold milk.

Many details match Foreigner by C.J. Cherryh (as per the comment by DannyMcG). Many details differ.

• Humans are stranded on an alien world. The main part of the story is several centuries after the landing, and the main character (Bren Cameron) is a trained diplomat, not a pilot. There is an introduction about how the humans got shipwrecked in the first place, which features bridge crew.
• The aliens are less advanced than the humans, but catching up quickly due to a land-for-tech deal. No longer nomadic.
• Outside the major towns, riding animals (mecheiti) are used which match your description.
• Cameron is taken captive by a faction of the aliens.
• Cameron is poisoned by alien tea. An alien physician, without access to human-specific drugs, prescribes a glass of milk every three hours until the milk no longer tastes bitter like the tea. They serve warm milk, but a half-dazed Cameron insists on cold milk.