Actually, my grounds for leaving the church were strictly religious (or non-religious, as it were) as well - the tax etc. issues are more like side-effects. (Though I will still continue to consider myself baptized. :))