Having the right doctrine, the right lifestyle and the right feelings toward God does not make one a Christian. Check out the words of a 17th century pastor. From Henry Scougal's, The Life of God in the Soul of Man.
I cannot speak of religion [the Christian faith - see note at bottom], but I must lament, that among so many pretenders to it, so few understand what it means: some placing it in the understanding, in orthodox notions and opinions; and all the account they can give of their religion is, that they are of this and the other persuasion, and have joined themselves to one of those many sects whereinto Christendom is most unhappily divided. Others place it in the outward man, in a constant course of external duties, and a model of performances. If they live peaceably with their neighbours, keep a temperate diet, observe the returns of worship, frequenting the church, or their closet, and sometimes extend their hands to the relief of the poor, they think they have sufficiently acquitted themselves. Others again put all religion in the affections, in rapturous hearts, and ecstatic devotion; and all they aim at is, to pray with passion, and think of heaven with pleasure, and to be affected with those kind and melting expressions wherewith they court their Saviour, till they persuade themselves they are mightily in love with him, and from thence assume a great confidence of their salvation, which they esteem the chief of Christian graces. Thus are these things which have any resemblance of piety, and at the best are but means of obtaining it, or particular exercises of it, frequently mistaken for the whole of religion...
But certainly religion is quite another thing, and they who are acquainted with it will entertain far different thoughts, and disdain all those shadows and false imitations of it. They know by experience that true religion is a union of the soul with God, a real participation of the divine nature, the very image of God drawn upon the soul, or, in the apostle’s phrase, "It is Christ formed within us."
Though most Christians today have an unbiblical aversion to the term "religion," the Puritans did not. They in fact believed there was only one true religion (the faith once for all delievered to the saints), and that everything else was simply false religion.