Heltorf is first documented as Hof Helethorpe in the register of Kaiserswerth Abbey from the 11th century. From 1662, Heltorf came into the possession of the Counts of Spee, who still own the castle today. The three-wing brick outer bailey was built in 1696. Rectangular towers flanking the gate tower in the centre are located at each of the western and eastern corners. The manor house, which was later built between 1822 and 1827 according to plans by Heinrich Theodor Freyse, is a classicist plaster building with a slightly protruding and gabled central risalit that extends over three storeys.
The so-called Garden Hall contains murals by the artists Karl Stürmer, Heinrich Mücke, Carl Friedrich Lessing and Hermann Freihold Plüddemann, who created a Barbarossa cycle between 1826 and 1841 in which the scenes are arranged in a theatre-like manner. Georg Dehio's Handbook of German Art Monuments describes the paintings as the ‘high point of late Romantic history painting by the Düsseldorf school of painters’. Directly next to the manor house is the castle chapel, built in the neo-romanesque style according to designs by Rudolph Wiegmann in 1854. On the opposite side of the building complex is the neo-gothic library, which was built in 1862 by the Cologne cathedral architect Vincenz Statz.
