The first Middleton Hall was built in the early 1600s.  Little is known about this old mansion except that it was built by Henry Middleton,  who was the High Sheriff of Carmarthenshire in 1644.  Both Henry`s son Christopher and his grandson Richard were High Sheriffs of the county in 1668 and 1701 respectively.  Richard Middleton`s son Henry died childless and Middleton Hall passed to his sister Elizabeth.  Elizabeth married Thomas Gwyn of Gwempa and their eldest son, Richard Gwyn, inherited the estate in the 1750s.  Richard Gwyn settled at Middleton Hall and was High Sheriff of Carmarthenshire in 1761.  He fell into debt and the estate was passed on to his eldest son Francis Edward Gwyn.  Francis lived beyond his means and in 1776 Middleton Hall was sold to pay off debts.

The estate passed into William Paxton`s hands in 1789 for about £40,000.    The old mansion was turned into a home farm.

Between 1793 and 1795 a new mansion, designed by Samuel Pepys Cockerell, was built at the heart of the estate.   The new Hall was a quadrangular mansion of neo-classical design, built in stuccoed brick with a grand ionic portico. The whole estate was re-landscaped into the fashionable feautiures of the day with the help of landscape architect Samuel Lapidge and engineer James Greer. The map painting, illustrated above, was made in 1815 of the whole Paxton Estate.  

On Paxton`s death in 1824 the Hall and estate were sold to Edward Adams.  Adams, a wealthy man who had made his money in Jamaica, paid £54,700.

On Edward Adams` death the estate passed to his son, also called Edward Adams.  He changed the family name to Abadam, hence the ‘Abadam Arms’ in nearby Porthyrhyd.

When Edward Abadam died in 1875 the estate passed to his daughter Adah.

In 1919 the estate was sold to Colonel William N. Jones.

In 1939 the lakes which had distinguished the estate in Paxton`s day were drained.  The dams were probably breached in accordance with new government legislation regarding reservoirs and their safety.

In 1931 the mansion was gutted by fire.

Carmarthenshire County Council bought the estate in the 1930s and divided it into seven farming starter units for lease.

In the 1954 the derelict walls of the mansion were pulled down.

In 1978 a scheme was set up to restore some parts of the park for public access.

In the mid 1980s painter William Wilkins conceived the idea of a National Botanic Garden of Wales.