Sir Thomas Rant (1606-1671) built four almshouses for poor widows on the village green. Now converted into two, they are still run as a charitable trust. He was probably also responsible for the large building known as Rant's or Thorpe Hall, which formerly stood near the church, and was demolished around 1780. This was his manor house. Very little is known of it except that it was probably built of red brick. Its walled garden survives behind the church, also largely of brick, and has features, such a polygonal corner turrets, which suggest that the house would probably have been more renaissance in appearance (such as Blickling Hall) than classical (like Hanworth Hall). In 1664 "Dame" Rant paid twenty shillings tax for twenty working hearths, whilst Gunton paid eleven, Hanworth paid ten and Felbrigg paid fifteen. It was a very substantial building, possibly similar in overall size and style to the hall at Barningham Winter.

Thorpe Hall's final occupant was Harbord Morden Harbord, later Lord Suffield, who was Sir Thomas's great-great-grandson. He inherited the estate at Gunton from his uncle, Sir William Morden Harbord, in 1770. Shortly afterwards he moved to Gunton and demolished Rant's Hall. Thorpe Market then effectively became part of Gunton Estate, much of it remaining in the ownership of the Harbord/Suffield family until the 1980s.

In 1796 Lord Suffield demolished the crumbling church at Thorpe Market and replaced it with the present building, rather as his uncle had done at Gunton thirty years previously. The resulting building has inspired mixed emotions from architectural writers ever since.
Throughout the nineteenth century Thorpe Market was run as a typical estate village, with large tenant farms which were the centre of the local economy. In the nineteenth century Thorpe Market gained a school (built in a rather similar style to the church), and what must surely be one of the remotest railway stations in Norfolk. Gunton Station, as it was known, was built primarily for the convenience of Lord Suffield (a major investor on the railway), and despite its rather splendid station buildings, was probably never heaving with local commuters.

A rather splendid gateway to Lord Suffield's estate was built in Thorpe Market in 1838. It's tall tower, from which Norwich cathedral is visible on a clear day, later served as a lookout for Royal trains arriving at Gunton Station. By the 1980s the tower was completely derelict, but has since been carefully restored.

Early in the twentieth century Thorpe Market gained a Post Office, a Methodist chapel and a Reading Room/Social Club. All three have since demised, although the former Reading Room is still in use as a Village Hall.