Richard Rigby, Secretary of Ireland, Royal Paymaster of the Forces, was a member of the Rigby family also known as Rigby of Mistley Hall in Essex, the site of their manor. Originally, the family was descended from the Rigby of Burgh family. His father and immediate ancestors made a fortune as merchant drapers in the city of London and as merchants and colonial officers in the West Indies, as well as in the South Sea Bubble speculation. Richard Rigby accumulated a fortune of his own serving the Crown and politician wheeler dealers in the dynamic 18th century parliament. Rigby spent much of his fortune reinvesting in the family seats of Mistley and Manningtree, employing the top architects and landscape artists of the day to build a port and spa, which failed. The ruins survive as a tourist destination. Though other members of the family continued to bear the Rigby name and arms, the bulk of Richard Rigby's wealth fell to his daughter who married Gen. Hale, and ultimately to the Pitt Rivers family whose members endowed the Pitt-Rivers museum at Oxford University.

Richard Rigby's father was also named Richard Rigby, and was the Secretary of Jamacia, the Provost Marshall, and a member of the Royal assembly on that island in the late 17th and early 18th century. At that time, the privateer trade was in its full glory as English sea captains raided the Spanish Main with letters of marque from the Queen (and King). The privateer trade closely overlapped with general piracy, and thus Jamaica became known as a pirate haven, especially under the leadership of Governor Morgan, himself a pirate. Port Royale of Jamaica was the headquarters of Caribbean piracy and privateering. Richard Rigby's brother, James Rigby, also served as a colonial officer on the island. Richard and James Rigby were sons of James Rigby of Mistley Hall, a London merchant, and Anne Hyde, a close cousin of Queen Anne (Hyde), Queen Anne, Queen Mary, and the Earl of Claredon.