The castle came under British ownership in the 1800s. At the height of the Trans Atlantic Slave Trade the castle housed 1000 male and female slaves and 300 slave masters.