Tangyue Village is probably most famous for its series of memorial archways, but also offers a number of historical buildings and temples. These archways do look impressive and should certainly be on your list when riding in the area.
At the time of my visit I was allowed to pass with my bike, but I am not sure this is always the case - probably depending on the size of the crowd.