It's possible the name might have misled you slightly, as Castle Philip is not actually a castle, but rather a dramatic rock outcrop perched just before the summit of Seefin.
Historically it served as a hideout for outlaws, but these days it's hikers along the Ballyhoura Way who seek refuge amongst the jagged boulders.