Short answer, no, it's not expensive to start composting. All you need is leftover plant material (uneaten food, bananna peels, cut flowers, etc) and someplace to put it. An enclosure for compost, however, can be done about as cheap or as expensively as you are able. Currently, we have 2 compost piles. We ran out of space in the original bin I built, and I just haven't got everything moved to the new pile or a new bin built. While we have some coons and such around here, so far we haven't noticed anything getting into the open pile.
The original bin I built was just made out of some particoard scraps leftover from the building of our house, while some holes drilled in the side for air circulation.
Only problem I've had so far is that my pile isn't getting hot enough to kill off the tomato and pumpkin seeds and such... Any ideas for that?