It's not mentioned in the books, but I think it's safe to assume the rules for magical duelling would be based on the Code Duello. (the link is American, but we were using essentially the same rules as Europe)
Code Duello: The Rules of Dueling
Reprinted from "American Duels and Hostile Encounters," Chilton Books,
1963.
The Code Duello, covering the practice of dueling and points of honor,
was drawn up and settled at Clonmel Summer Assizes, 1777, by
gentlemen-delegates of Tipperary, Galway, Sligo, Mayo and Roscommon,
and prescribed for general adoption throughout Ireland. The Code was
generally also followed in England and on the Continent with some
slight variations. In America, the principal rules were followed,
although occasionally there were some glaring deviations.
Rule 1. The first offense requires the first apology, though the
retort may have been more offensive than the insult. Example: A tells
B he is impertinent, etc. B retorts that he lies; yet A must make the
first apology because he gave the first offense, and then (after one
fire) B may explain away the retort by a subsequent apology.
Rule 2. But if the parties would rather fight on, then after two shots
each (but in no case before), B may explain first, and A apologize
afterward.
N.B. The above rules apply to all cases of offenses in retort not of
stronger class than the example.
Rule 3. If a doubt exist who gave the first offense, the decision
rests with the seconds; if they won't decide, or can't agree, the
matter must proceed to two shots, or to a hit, if the challenger
require it.
Rule 4. When the lie direct is the first offense, the aggressor must
either beg pardon in express terms; exchange two shots previous to
apology; or three shots followed up by explanation; or fire on till a
severe hit be received by one party or the other.
Rule 5. As a blow is strictly prohibited under any circumstances among
gentlemen, no verbal apology can be received for such an insult. The
alternatives, therefore -- the offender handing a cane to the injured
party, to be used on his own back, at the same time begging pardon;
firing on until one or both are disabled; or exchanging three shots,
and then asking pardon without proffer of the cane.
If swords are used, the parties engage until one is well blooded,
disabled, or disarmed; or until, after receiving a wound, and blood
being drawn, the aggressor begs pardon.
N.B. A disarm is considered the same as a disable. The disarmer may
(strictly) break his adversary's sword; but if it be the challenger
who is disarmed, it is considered as ungenerous to do so.
In the case the challenged be disarmed and refuses to ask pardon or
atone, he must not be killed, as formerly; but the challenger may lay
his own sword on the aggressor's shoulder, then break the aggressor's
sword and say, "I spare your life!" The challenged can never revive
the quarrel -- the challenger may.
Rule 6. If A gives B the lie, and B retorts by a blow (being the two
greatest offenses), no reconciliation can take place till after two
discharges each, or a severe hit; after which B may beg A's pardon
humbly for the blow and then A may explain simply for the lie; because
a blow is never allowable, and the offense of the lie, therefore,
merges in it. (See preceding rules.)
N.B. Challenges for undivulged causes may be reconciled on the ground,
after one shot. An explanation or the slightest hit should be
sufficient in such cases, because no personal offense transpired.
Rule 7. But no apology can be received, in any case, after the parties
have actually taken ground, without exchange of fires.
Rule 8. In the above case, no challenger is obliged to divulge his
cause of challenge (if private) unless required by the challenged so
to do before their meeting.
Rule 9. All imputations of cheating at play, races, etc., to be
considered equivalent to a blow; but may be reconciled after one shot,
on admitting their falsehood and begging pardon publicly.
Rule 10. Any insult to a lady under a gentleman's care or protection
to be considered as, by one degree, a greater offense than if given to
the gentleman personally, and to be regulated accordingly.
Rule 11. Offenses originating or accruing from the support of ladies'
reputations, to be considered as less unjustifiable than any others of
the same class, and as admitting of slighter apologies by the
aggressor: this to be determined by the circumstances of the case, but
always favorable to the lady.
Rule 12. In simple, unpremeditated recontres with the smallsword, or
couteau de chasse, the rule is -- first draw, first sheath, unless
blood is drawn; then both sheath, and proceed to investigation.
Rule 13. No dumb shooting or firing in the air is admissible in any
case. The challenger ought not to have challenged without receiving
offense; and the challenged ought, if he gave offense, to have made an
apology before he came on the ground; therefore, children's play must
be dishonorable on one side or the other, and is accordingly
prohibited.
Rule 14. Seconds to be of equal rank in society with the principals
they attend, inasmuch as a second may either choose or chance to
become a principal, and equality is indispensible.
Rule 15. Challenges are never to be delivered at night, unless the
party to be challenged intend leaving the place of offense before
morning; for it is desirable to avoid all hot-headed proceedings.
Rule 16. The challenged has the right to choose his own weapon, unless
the challenger gives his honor he is no swordsman; after which,
however, he can decline any second species of weapon proposed by the
challenged.
Rule 17. The challenged chooses his ground; the challenger chooses his
distance; the seconds fix the time and terms of firing.
Rule 18. The seconds load in presence of each other, unless they give
their mutual honors they have charged smooth and single, which should
be held sufficient.
Rule 19. Firing may be regulated -- first by signal; secondly, by word
of command; or thirdly, at pleasure -- as may be agreeable to the
parties. In the latter case, the parties may fire at their reasonable
leisure, but second presents and rests are strictly prohibited.
Rule 20. In all cases a miss-fire is equivalent to a shot, and a snap
or non-cock is to be considered as a miss-fire.
Rule 21. Seconds are bound to attempt a reconciliation before the
meeting takes place, or after sufficient firing or hits, as specified.
Rule 22. Any wound sufficient to agitate the nerves and necessarily
make the hand shake, must end the business for that day.
Rule 23. If the cause of the meeting be of such a nature that no
apology or explanation can or will be received, the challenged takes
his ground, and calls on the challenger to proceed as he chooses; in
such cases, firing at pleasure is the usual practice, but may be
varied by agreement.
Rule 24. In slight cases, the second hands his principal but one
pistol; but in gross cases, two, holding another case ready charged in
reserve.
Rule 25. Where seconds disagree, and resolve to exchange shots
themselves, it must be at the same time and at right angles with their
principals, thus:
If with swords, side by side, with five paces interval.
N.B. All matters and doubts not herein mentioned will be explained and
cleared up by application to the committee, who meet alternately at
Clonmel and Galway, at the quarter sessions, for that purpose.