In Dexter's lab there is a constant rivalry between Dexter and Mandark. They try to outdo each other over and over again. In any episode, was it ever determined who was actually the smarter person? (before we get into how do you determine smart/intelligence/etc, I'd like to use the in world rules for intelligence since I remember they did speak in terms of relative or absolute intelligence readings.
If you go by the first introduction of Mandark in the Episode "Dexter's Rival", Mandark is smarter.
When Mandark started going to Huber Elementary School on the first day of the school year, he would end up meeting his future rival Dexter and as soon as they came face to face, Mandark proved that he was the far superior intellect. Upon their first meeting, Mandark proved that he was far smarter than Dexter and even won over the affections of Dexter's beloved teachers. To try and prove that he was still superior, Dexter showed Mandark his laboratory thinking it would mean that he was smarter than him, but then Mandark revealed that he had an even bigger and better laboratory. Mandark then forced Dexter to shut down his lab, or else become Mandark's lackey, and Dexter, unable to let go of his pride, shut down his laboratory.
Yes, Dexter, I can read your thoughts, and I am smarter than you.
-Mandark ("Dexter's Rival")
Not a real answer since you want to know if Dexter or Mandark is the smarter one but there was an episode in which Dee Dee became more intelligent than Dexter. This was the Episode "Dexter's Assistant" (emphasis mine, also I'm only adding this as an answer because it's one of my favourite episodes):
Dexter puts the brain inside of Dee Dee and she becomes smart. Dexter takes Dee Dee over the the transportation machine again and he gets in. Dee Dee tells Dexter she doesn't think this will work but he ignores her warning, starts the machine and tells her to push the button which she does. However the machine messes up and blows Dexter off from the top and shutting the power down again.
Later, Dexter is working with fuses and he has a bunch of long flowing hair. Dee Dee tells him what fuses are right and what fuses are wrong but Dexter tells her that he knows what he is doing. Dexter cuts the wron wire and it explodes in his face. Dee Dee tells him that they were all wrong.
Later, Dexter is at the science fair and shows his great invention. The winner of the science fair gets announced and the winner is said to be Dee Dee who has built a Merry-Go-Round machine...