I am going to propose as an answer "Aristotle and the Gun" by L. Sprague de Camp, Astounding Science Fiction February 1958. Also published in the British edition of that magazine, May 1958.

I'm including a plot summary from Wikipedia at the end of this answer. There are some points of disagreement with your question, which I discuss shortly.

Points of agreement are:

1. written in the form of a letter to a doctor/professor. Wikipedia: "The narrative of the story is set forth by Weaver in a lengthy letter to an acquaintance". As the writer is a scientist, his acquaintance could well be a doctor (PhD, not medical) or a professor.
2. the writer is a scientist. Wikipedia: "The lonely and misanthropic scientist Sherman Weaver".
3. The society is suspicious of them. Wikipedia: "Weaver pretends to be a conventional traveler from India. Equipped with modern-day marvels, he attempts to demonstrate to his new acquaintance Aristotle the value of experimentation in furtherance of knowledge. His task is complicated by the malicious mischief of Aristotle's students, the coterie of young Prince Alexander (subsequently Alexander the Great), and by coming under suspicion of being a spy for the Great King of Persia, against whom Philip is preparing to go to war." Note the final two clauses.
4. they interact a little via gifts. Wikipedia: "Equipped with modern-day marvels, he attempts to demonstrate to his new acquaintance Aristotle the value of experimentation in furtherance of knowledge." I can't remember whether the protagonist donates any of these marvels to Aristotle: if not, agreement is lessened here.
5. Then somehow something terrible happens related to escalating tension with the society (I think the writer and his friends repeatedly do research or use items that are considered forbidden to the society). Wikipedia: "Ultimately forced to defend himself with a handgun he has brought, Weaver is on the point of being executed for espionage and murder". I felt a sense of escalating tension. The writer is not doing research, but he is constantly trying to teach Aristotle in order to advance ancient Greek science.
6. I think there was a gun involved. See title! I weighted this part of your question highly when asking myself whether I should propose "Aristotle and the Gun" as an answer. Likewise, your saying that the story was a series of letters.

Points of disagreement are:

1. describes an experience with a time machine with himself and another man. There is only one time traveller. He doesn't travel with a friend, although he does later become as friendly with Aristotle as his misanthropic personality would probably allow him to be with anybody.
2. the writer's friend ends up dying. Impossible, since there is no friend. So perhaps I am not giving the right answer.

Also, this kind of innocent-traveller-in-the-past scenario is fairly common, so there may be many other candidates. See for example Poul Anderson's "The Man who Came Early".

Plot summary

The lonely and misanthropic scientist Sherman Weaver has a central role in a secret US Government project to build a time machine. The project succeeds and a prototype device is constructed. But before it can be tested, the government - alarmed at Weaver's report that small changes in history might have profound consequences and completely change the present day world - decides to abort the project. Weaver is ordered to dismantle the machine. Rather than obey, he takes matters into his own hands, using the machine to project himself back to the era of Philip II of Macedon. There he hopes to meet Aristotle. Believing that the influential ancient philosopher's lack of interest in experiment retarded scientific progress through much of subsequent history, Weaver aims to nudge the savant in what he considers the proper direction. His intention is to create a different Twentieth Century dominated by super-science, hundreds of years in advance of ours.

Weaver pretends to be a conventional traveler from India. Equipped with modern-day marvels, he attempts to demonstrate to his new acquaintance Aristotle the value of experimentation in furtherance of knowledge. His task is complicated by the malicious mischief of Aristotle's students, the coterie of young Prince Alexander (subsequently Alexander the Great), and by coming under suspicion of being a spy for the Great King of Persia, against whom Philip is preparing to go to war. Ultimately forced to defend himself with a handgun he has brought, Weaver is on the point of being executed for espionage and murder when he is snapped back into the present day as the effects of his time projection wear off.

Weaver finds himself in a world very different from the one he left, but not in the way he hoped. Aristotle, convinced that the tedious accumulation of experimental knowledge is beneath the dignity of civilized philosophers and that it is a waste of time attempting to catch up to "India" in that regard, turns out to have come down strongly against the notion in his writings. The result is a backward present of petty states, roughly at the level of late Medieval principalities in our own history, considerably behind Weaver's original timeline in technology. His own United States is not even a dream, its physical confines being controlled by various Amerindian nations, influenced by the civilization of the Old World but having long since thrown off any subjection to it. Enslaved in one such state, Weaver is only delivered from endless drudgery after many years, when his scholarly talents are finally recognized.

The narrative of the story is set forth by Weaver in a lengthy letter to an acquaintance curious as to his remarkable background, in which he concludes that he would have done better to leave well enough alone.