Per the official trail guide to the River to River Trail, this church often has a spigot around back that thru-hikers are welcome to use to fill up their water supplies. That said, if in doubt as to the rules, always be polite and ask first.
This also marks the start (or end) of about a mile of road walking, depending on which direction you're headed.