Ruling on the difference between the follower's intention and the imam's intention

If a follower joins the imam intending to pray Maghrib while the imam is praying 'Isha', or joins intending to pray 'Asr while the imam is praying Zuhr, is there any issue with that?
A difference in intention between the imam and the follower does not cause harm; for Mu'adh ibn Jabal used to pray a voluntary prayer with his people while they were praying an obligatory prayer, and this is one of the greatest forms of difference, yet the Prophet (peace and blessings be upon him) did not disapprove of it.