As a freelancer and in my experience paying freelancers, the hourly rate agreed stands wether it's cash or transfer.
It's up to you to insist that payment from your company needs to be made by transfer, or that a receipt needs to be supplied for accounting purposes.
This then makes the 'cash payment' less of an incentive as it's then less likely the person will use it for tax avoidance or benefit fraud.
If they then insist on cash without a paper trail for whatever reason, it's up to you to morally and legally decide what to do. At this point you could bring up the concept of a 'cash discount'. This could be to the value of whatever tax you'd assume they would be paying.
If they operate as a limited company that is VAT registered, the difference represents 33% off from what you would have paid before. (100%-1/6th VAT then minus 20% corp tax = 67%)
There is less benefit to you to do this if you are already vat registered as you'd be claiming this VAT back anyway.
If they are a sole trader or self employed they should be paying on average about 20/40% tax (but probably about 20% if they are cheap enough to try for cash payments) so you could expect a 20% discount as reasonable in this case. (Another rationale is that if you are paying it into your bank account to pay them, you should take off the tax that you will be paying on it before getting the cash out for them, so if you pay 20 or 40% tax on that money that would perhaps be the discount.)
Finally if they are claiming benefits, and are fully in the 'benefit trap' almost all the money they earn (0r maybe 80% of it) would be taken from the money they are receiving from the government. For them there would be little incentive to do any 'part time' work that wasn't cash. If this is the case, ask yourself 'Am I already getting this service at a much discounted rate anyway?' Perhaps they originally quoted you expecting to ask for cash?
Personally if it's just a small amount and you're getting the service cheaply, I'd just find a way to pay it without quibbling, and not be a miser.
If it's a large amount and you're only comfortable doing things exactly by the book, then I would insist on making them a transfer and having an invoice.
The middle ground is casually asking for a cash discount, paying them in cash and asking them to invoice you anyway. Then you have the paperwork and the burden is on them to report and account for the money correctly.
Please don't screw this person over for money just because you've seen an opportunity to exploit them.
Finally, please don't haggle on the fee after the service has been rendered, that is not classy at all.
p.s. I hope this answer was useful to you. this answer is for hypothetical purposes only, always stay informed and stay legal! This is not a substitute for advice from an accountant!