While I much prefer the look of clay pots, especially those that have been glazed and painted, for starting seeds or small plants I would use plastic, or like
@gata montes says, any kind of container will work well, like yogurt, sour cream, cottage cheese containers, milk or juice jugs, etc. Just poke a few holes in the bottom so they will drain. My mom saves these types of containers all year and then uses them in spring. I also save the pots and trays that all my new plants come in. If you save money on these containers, you'll then have more money to spend on the plants when they are grown and you want a "container garden".
One thing to keep in mind with clay or ceramic pots - if you live where it freezes, make sure you turn any empty pots upside down, because if they get water in them and then freeze, they will crack and break.