I went to Gaya during my round the world trip and I experienced one of the most difficult parts of my travels in India. The train times are totally crazy and regularly run up to 12 hours late. I came across one traveler who'd been waiting for his for 62 hours! I arrived in the middle of the night with no hotel reservation and the only place I could find to sleep was the freezing construction site for a guest-house. To avoid the same problems as I had, I advise you to book ahead.
At daybreak I saw that Gaya is hideous. But, since it's just a stop on the way to Bodhgaya, it doesn't really matter. 12 km further on and I felt like I'd gone from hell to heaven. Mahabodhi Temple is classed as a World Heritage Site by UNESCO and is a calm, peaceful (a rare thing in India, which makes it even more attractive), and sacred place for Buddhists. It was here, under the sacred banyan tree, that Siddartha received enlightenment and became a Buddha. The ghastliness of Gaya is soon forgotten here.
I only had a short stop at Gaya, whilst travelling from Patna to Bodhgaya and I suggest you do the same. Take care when travelling around Gaya, as petty crime is rife. It's best to arrive and leave during the day.
Surprisingly, Gaya is an important pilgrimage site for Hindus! You can go and see the pink outside of the Vishnupad Temple, but foreigners are forbidden to enter.
It's best to head straight to Bodhgaya (at 13km from Gaya) and the Mahabodhi Temple, where the Nepalese Prince Siddhartha Gautama attained enlightenment. There's also a giant Buddha and Buddhist temples from other countries in the Southeast Asia.