Around here, we don't have many horse-riding people. There are, however, a few riding stables in adjoining counties that offer riding lessons and horse rentals.
Pathfinder1 is right. There are plenty of books in your local library that will tell you quite a bit about horses and training. But you need to find a stable and there's plenty of those in GA. Go for it.
The best thing to do is take lessons. I would also suggest picking up a couple of horse magazines when you get a chance. You will need to decide whether you want to ride Western or English.
Go to the library and get some books on the subject. There are lots of good videos now. RFD-TV has plenty of helpful shows on how to ride and how to solve problems.
Taking lessons is a great idea. Riding can be as sophisticated as you want it to be. Done well it is very subtle and involves a lot more than kick them to go and pull the reins to stop.
One of the unintended consequences of working with horses is that is makes you a better human. Horses are prey animals and have great fear. Your job is to make them feel safe and then they will work for you. Horses have their own language and it is all about how you move your body and how you touch them.
There is also a serious caution that comes with riding. Equines are large animals that can hurt you. Riding involves some risk. Be carefu what you swing your leg over. Most horses are time bombs waiting to go off. They are prey animals and when they get scared they have a flight response.
Take lessons and maximize your safety. People that work with horses a lot get hurt. It takes some time to learn how to be safe. Avoid going over to ride your friend's horses. They can be some of the most dangerous. Green on green equals black and blue. Find some older, calmer horses to ride.