The museum was founded in 1971 by the Pacificulture Foundation, which purchased "The Grace Nicholson Treasure House of Oriental Art" from the City of Pasadena. Grace Nicholson donated the structure to the city for art and cultural purposes in 1943 and was a dealer in Native American and, later, Asian art and antiques. It houses some 15,000 rare and representative examples of art from throughout Asia and the Pacific Islands. In 2013, the museum became part of the University of Southern California.
The building, which is listed in the National Register of Historic Places, was built in 1926 and designed by the architectural firm of Marston, Van Pelt & Maybury. It is designed in the style of a Chinese imperial palace and features a central courtyard with a garden, a small pool, and decorative carvings.