If that’s all you’ve got for leverage, the point is made.
Besides, that hole would not be “big.” A vast difference between Britain’s budgetary contributions and its receipts was a fiction perpetrated by proponents of Leave. Also, Britain is still pledged to remit to the EU a significant amount of money, however the saga ends.
The Irish are the only ones seriously “worried,” but that worry extends to everyone in Northern Ireland — and that worry is purely a product of this desire on the part of some to secede from the Union.