This is probably P. L. Travers's book series about the magical nanny Mary Poppins, of which the first two books are Mary Poppins (1934) and Mary Poppins Comes Back (1935).
Mary Poppins is the babysitter to the Banks family. She takes care of Jane and Michael (who are probably just below school age), John and Barbara (who turn one year old at the end of the first book), and Annabel (born during the second book). The adventures often take place in streets of the town that appear only when the children visit them with the help of Mary Poppins and aren't there later. The children visit relatives and acquaintances of Mary Poppins who all seem to possess some sort of magical powers or curses. Mary Poppins herself indeed seems to be important in the fantasy world. In the first book, the children visit the zoo at night, when humans are in the cages, the zoo animals are outside, and they are celebrating Mary Poppins's birthday. In the second book, they instead visit a circus in the sky at night in which constellations behave like living people and perform, the Sun is the director, and the Sun dances with Mary Poppins. The parents don't know or don't believe about the fantastical adventures, because Mary Poppins always acts in front of them like nothing happened. Mary Poppins is indeed very strict, but the start of the second book makes it clear that the children prefer her over any other babysitter that they had. The parents may be skeptical at the first book, and certainly don't like how Mary Poppins leaves without advance notice at the end of both books, but they do recognize that Mary is good with the children. The brown hair doesn't match: Mary Poppins has black hair according to the first chapter of Comes Back.
