One of the main towns in Borneo, populated by around 150,000 inhabitants, was for me a simple stopover when I travelled in the east of Sabah. This coastal town doesn't have any beaches, and is completely surrounded by oil palm plantations. Lahad Datu lacks attractions and interests despite a decent tourist development.
It effectively seems like a strategic point to organise a passage through on the the nature reserves in the region, situated only a few hours away by bus or taxi. Whether it's to discover the tornadoes of barracudas in Sipadan or the smallest elephants in the world at the Tabin reserve, you will find a tourist information centre as well as specialised agencies to give you information and reserve an excellent service during your stay in Malaysia.
I would therefore recommend that you get information and reserve as quickly as possible (booking in advance is generally mandatory) and to not delay time in this town, consider it as a simple stop.