Although Püspökszentlászló is already mentioned in a 13th-century charter, the village became deserted during the Ottoman occupation. Only at the end of the 18th century did it become populated again. In 1725, the Minorites of Pécs erected a small chapel here. In 1797, Pál László Esterházy, Bishop of Pécs, built a church in its place and a summer residency right next to it, in the late Baroque style. Since Joseph II dissolved the Pauline order, it is presumed that the bishop – himself a former Pauline monk – intended the mansion to serve a camouflaged monastery. The division of the edifice seems to support this theory: on the first floor, cell-like rooms were created. However, the Paulines did not move into the building; it stood empty for nearly a hundred years. In 1898, Bishop Sámuel Hetyei renovated the building and established an arboretum around it. The renovation works were completed by Bishop Count Gyula Zichy a few years later. He joined the mansion with the church next to it. The Angster organ, functioning to this day, was also placed there at the time.
Following the Second World War, the nuns of the Ursuline Convent, teaching in the village school, lived in the building. After the dissolution of religious orders, it served as a care home for sisters. Late in the summer of 1955, Cardinal József Mindszenty was kept under house arrest for about three months in the mansion.
