Today's museum, a mighty 3-storey palace building, was built in the years 1753 - 1755 as the provost of the canons of St. Margaretha by the architect Ludwig Oswald from Villingen. The Margaret statue on the central gable of the Probstei is the work of the sculptor Josef Anton Hops from Villingen. The plasterer Franz Anton Vogel from Freiburg created the stucco ceilings.
Between 1815 and 1873 the building was used as a cotton weaving mill. Later converted into a hotel, the former provost's house saw such distinguished guests as Kaiser Wilhelm I and his family in 1880. From 1891 to 1977, the palace building housed different types of schools. After extensive restoration, the Elztal Museum with its extensive collection was opened to the public in 1985 in the baroque palace.