What is the ruling on one who concludes the prayer after two rak'ahs of the Dhuhr prayer, assuming it was the Fajr prayer, and then remembers?

The Shaykh : Now, what is your opinion regarding a man who concluded the prayer after two rak'ahs of the Dhuhr prayer, assuming it was the Fajr prayer, and then remembered? Does he complete the two rak'ahs, or does he start over? Consider it so that we may reach a conclusion... Does he build upon what he has done and complete the two rak'ahs and perform the prostration of forgetfulness, or does he start the prayer over? They said that he must start over because he concluded the prayer based on the assumption that his prayer was two rak'ahs, whereas he was in the state of a four-rak'ah prayer and concluded after two rak'ahs. For this reason, they say that in this case, it is obligatory for him to start the prayer over because he concluded it assuming it was a two-rak'ah prayer only.