I've got a short story here that quite matches your description. It's by Herbert W. Franke and appeared in his (IMHO excellent, stunning, and ageless) collection "Der grüne Komet", chapter 46 "Das Gebäude" (for the non-german readers: "The green comet" - "The building"). I don't know if the collection has ever been translated.
Die blaue Sonne war unter den Horizont gesunken, die rote Sonne stieg siegreich empor. [...] Fontain war Maurer. Das heißt, er durfte die Steine übereinanderschichten, die die Träger aus den Feldern herbeischleppten. [...] Inspektionsroboter strichen unablässig hinter den Reihen der Schaffenden vorbei. Erst in ihrer Freizeit, in den wenigen Stunden der blauen Nacht, die ihnen vor dem Schlafen blieben, unterhielten sie sich darüber – über das Gebäude das sie errichteten [...] Das Gebäude erstreckte sich weit nach allen Richtungen; noch keiner hatte die ganze Insel gesehen [...] Das violette Leuchten über der Stadt zeigte den Morgen an – die letzten roten Strahlen verblaßten, das Blau des Tages breitete sich aus. [...] Hassan war Arbeiter. Das heißt, er meißelte Steine von den Mauern herunter. [...]
English translation (using Google translate and some manual improvements):
The blue sun had sunk below the horizon, the red sun rose victoriously. [...] Fontain was a bricklayer. That is, he was allowed to stack the stones, which were carried out of the fields by the bearers. [...] Inspection robots were passing behind the ranks of the creators. It was only in their spare time, during the few hours of the blue night that remained before sleeping, that they talked about it – the building they built [...] The building stretched far in all directions; no one had yet seen the whole island [...] The violet glow over the city was heralding the morning – the last red rays faded, the blue of the day spread. [...] Hassan was a worker. That is, he cut stones from the walls. [...]
