Faith in the heart is not sufficient without prayer etc. Rather it is obligatory to believe in one’s heart that Allah is One, with no partner or associate, and that He is one’s Lord and Creator, and to devote worship to Him alone, may He be glorified and exalted. And it is obligatory to believe in the Messenger Muhammad (blessings and peace of Allah be upon him) and that he is truly the Messenger of Allah to all of the two races (mankind and the jinn). All of this is essential, for it is the foundation of the faith. It is also obligatory for the accountable individual to believe in everything that Allah and His Messenger have told us about Paradise and Hell, the Siraat (bridge over Hell), the Balance and other things referred to in the Holy Qur’aan and saheeh Sunnah. In addition to that, it is essential to utter the declaration of faith (shahaadah), testifying that there is no God but Allah and that Muhammad is the Messenger of Allah, and it is essential to pray and to do all the other things enjoined in Islam. If the individual prays, then he has done what he is obliged to do, but if he does not pray then he has committed an act of kufr (disbelief), because not praying is kufr.
With regard to zakaah, fasting, Hajj and all the other obligatory matters, if he believes that they are obligatory but he is too lazy to do them, then he is not a kaafir because of that; rather he is disobedient or sinning, and his faith is weak and lacking, because faith may increase and decrease. Faith increases when one does acts of worship and righteous deeds, and it decreases when one commits acts of disobedience and sin, according to Ahl as-Sunnah wa’l-Jamaa‘ah.
With regard to prayer in particular, not praying is kufr according to many scholars, even if one does not deny that it is obligatory. This is the more correct of the two scholarly opinions. This is in contrast to other acts of worship such as zakaah, fasting, Hajj and so on. If he does not do them, that is not major kufr according to the correct opinion, but it detracts from and weakens faith, and it is a serious major sin. Not giving zakaah is a serious major sin; not fasting is a serious major sin; not performing Hajj when one is able to do so is a serious major sin – but it is not major kufr if one believes that zakaah is a duty, fasting is a duty, Hajj is a duty for the one who is able to do it. So long as one does not deny that these duties are obligatory, but he is negligent concerning them, then he is not a kaafir according to the correct opinion.
With regard to prayer, if he does not do it, then according to the more correct scholarly opinion he is a kaafir in the sense of major kufr –Allah forbid – even if he does not deny that it is obligatory, as stated above. That is because the Prophet (blessings and peace of Allah be upon him) said: “Between a man and shirk and kufr there stands his giving up prayer.” Narrated by Muslim in his Saheeh. And he (blessings and peace of Allah be upon him) said: “The covenant that differentiates between us and them is prayer; whoever does not pray is a kaafir.” Narrated by Imam Ahmad and the authors of the four Sunans with a saheeh isnaad. This applies equally to both men and women. We ask Allah to keep us safe and sound.
[Source] Shaykh ‘Abd al-‘Azeez ibn Baaz (may Allah have mercy on him)