Stormed and rebuilt several times, the palace subsequently experienced a chequered history. It was finally blown up in 1702 during the War of the Spanish Succession and served as a quarry for the next two centuries. At the beginning of the 20th century, the first restoration work began, which has continued to the present day with significant support from the Kaiserpfalz Kaiserswerth e.V. association.