Amazing site.
From Huangduobao you can reach them along the main road: panoramic but with more elevation and traffic; or a cycle path along the river (added as highlight as well).
The entrance to the caves is 48 yuan (in 2023) and 20 for the bus but you don't need the latter. It takes from the ticket office to the museum then to the caves and back to the beginning all on the main road.
You hit the ticket office first, get your ticket and cycle further in the main road 500m to the museum, then another 500m turn right across the bridge and park there to visit the caves.