It depends on the grade of wood you are starting with and how you plan on treating the finished product. If you are going to varnish it, start with a 120 grit to smooth the wood and always work with the grain. Then move to a 200 grit, then a 300.
Of course, you haven't told us the type of wood you are using (pine, maple, oak walnut, etc.). It will make a difference and these are only general guidelines. If you're working with briar, it's a different game altogether.