A poor person who was given from the zakah, then inherited wealth: Is the zakah to be taken back from him?

Question : Based on that, I gave to a poor man. The time for the zakah came and he was poor, so we gave him from the zakah. A relative of his died and he became wealthy, inheriting from him and becoming wealthy. We gave him ten thousand riyals from the zakah, and after three or four days his relative died and left behind millions of dirhams. This man became wealthy, so regarding the ten thousand, it is obligatory for him to give it to whom it is obligatory for me to give it.

Answer : It is not permissible, because he owns it, , he owns it now, so it is his.