Built in 1470 by Duke Pirro del Balzo, it is an imposing building, with a square plant with four cylindrical towers. The coat of arms of the Del Balzo family, the radiant sun, is visible on the west tower. The building of a Castle with a large moat led to the demolition of the Romanesque cathedral and the neighborhood surrounding it. Fortress, was transformed into a mansion by the Princes Carlo and Emanuele Gesualdo, with the addition of the loggia of the north wing and the inner-west. From 1612 it was the seat of the Academy of Rinascenti. Inside, there is a large courtyard surrounded by a Renaissance loggia. The rooms made in the base of the towers houses the National Archaeological Venosa Museum.