The most obvious example is Finwë, High King of the Noldor. His first wife, Míriel, died after the birth of their child (Fëanor, who you may have heard of)1. Shortly after this, he married Indis, of the Vanyar, who bore him four more children. From the published Silmarillion:
[I]n the bearing of her son Míriel was consumed in spirit and body; and after his birth she yearned for release from the labours of living. And when she had named him, she said to Finwë: 'Never again shall I bear child; for strength that would have nourished the life of many has gone forth into Fëanor.'
Then Finwë was grieved, for the Noldor were in me youth of their days, and he desired to bring forth many children into the Miss of Aman; and he said: 'Surely there is healing in Aman? Here all weariness can find rest.' But when Míriel languished still, Finwë sought the counsel of Manwë, and Manwë delivered her to the care of Irmo in Lórien. At their parting (for a little while as he thought) Finwë was sad, for it seemed an unhappy chance that the mother should depart and miss the beginning at least of the childhood days of her son.
Now it came to pass that Finwë took as his second wife Indis the Fair. She was a Vanya, close kin of Ingwë the High King, golden-haired and tall, and in all ways unlike Míriel. Finwë loved her greatly, and was glad again. But the shadow of Míriel did not depart from the house of Finwë, nor from his heart; and of all whom he loved Fëanor had ever the chief share of his thought.
The Silmarillion III Quenta Silmarillion Chapter 6: "Of Fëanor and the Unchaining of Melkor"
Finwë's desire to re-marry was actually a cause of some consternation in Aman, but I talk about that more elsewhere on this site.
I want to address concerns that this was in some way a special case; it wasn't. It was exceptional for two reasons only:
It was the first. This was the first time in Elvish history that a widowed Elf had expressed a desire to re-marry. Because it was the first, it required a judgement from the lawgivers of the Elves. That the lawgivers happened to be the Elves' gods2 isn't relevant
It took place in the Undying Lands. The Elves weren't supposed to die in the Undying Lands, that was kind of the point. The fact that Míriel did caused a bit of a philosophical ruckus that's far too complicated to get into.
It's not entirely accurate to say that Finwë was given permission to remarry; it would be more correct to say that his desire identified a gap in the laws, and it was ruled that this desire was just, under certain circumstances (most notably that the departed had to agree to never return to life as long as their ex-spouse was alive; Elves are a little weird like that).
There are only two other examples I can think of:
Túrin I, sixth of the Ruling Stewards of Gondor, is noted in The Peoples of Middle-earth as being quite unusual for having married twice:
[Túrin I] was wedded twice and had several children (a thing already rare and remarkable among the nobles of Gondor); but only the last, a child born in his old age, was a son.
The History of Middle-earth XII The Peoples of Middle-earth Chapter VII: "The Heirs of Elendil"
Of arguable canonicity is Ottor Wæfre (later called Eriol), a fifth-century Saxon who stumbles on the isle of Tol Eressëa, where the Elves tell him their ancient legends; this was Tolkien's framing device for The Book of Lost Tales, the earliest draft of what would come to be called The Silmarillion:
Ottor Wæfre settined on the island of Heligoland in the North Sea, and he wedded a woman named Cwén (Old English: 'woman', 'wide'); they had two sons named 'after his father' Hengest and Horsa 'to avenge [Ottor's father]' (hengest is another Old English word for 'horse').
Then sea-longing gripped Ottor Wæfre: he was a son of Eärendel, born under his beam. If a beam from Eärendel fell on a child new-born he becomes 'a child of Earendel' and a wandered. (So also in The Cottage of Lost Play Eriol is called by the author and by Lindo a 'son of Eärendel'.) After the death of Cwén Ottor left his young children.
History of Middle-earth I The Book of Lost Tales, Part One Chapter 1: "The Cottage of Lost Play" Notes and Commentary
Ottor later goes on to marry Naimi, a niece of Vairë3:
In Tol Eressëa [Ottor/Eriol] wedded, being made young by limpe4 (here also called by the Old English word lip), Naimi (Éadgifu), niece of Vairë, and they had a son named Heorrenda.
History of Middle-earth II The Book of Lost Tales, Part Two Chapter 6: "The History of Eriol or Ælfwine and the End of the Tales"
1 I'm simplifying massively. What actually happened is that Míriel was so wearied by giving birth to Fëanor that she went to dwell in Lórien (the region of Aman, not Lothlórien of Middle-earth) for healing. But she didn't find a whole lot of relief there, so she decided to depart from her body and live in Mandos as an unhoused spirit.
2 Again, vastly oversimplifying. Although maybe not that vastly, depending on how you look at it
3 Not to be confused with the Vala Vairë; they are different
4 Limpe is an Elvish beverage fairly similar to the Greek ambrosia; the word literally translates to "fairy drink", and is said to heal all ills and is strongly implied to have made Eriol immortal