I think a lot of people get hung up on the difference between a need and a want. The title of the post is do I 'need' a diesel. If you are towing "a couple times a year" as the original post says, I'd definitely put it in the category of want instead of need. If you are pulling a trailer for a living (i.e. contractor, dedicated race rig, concessions trailer, etc), I'd definitely say that a diesel is a better fit than a gasser. Even if the rig isn't used often, but when it is, it is to pull a heavy load, the diesel is 'typically' built with this in mind and will out pull and out last a gasser.
The real question is what percentage of its anticipated life will be spent pulling a heavy trailer, vs. how much of the time it will be spent pleasure driving. I have already lost count of the number of diesel rigs I've seen with standard tags (not weighted) which tells me the only thing they are meant to pull is a flat-bill wearing kid around the parking lot of the local mall.
For what you are describing, just about any 3/4 or 1 ton truck will suit you, the better question is 2WD or 4WD.
I've always wondered about putting a selectable locker (OX, or ARB) in the back of a 3/4 ton 2WD truck to get the best bang for the buck in an occasional tow rig. Since most of the time, you are towing on paved roads, the 2WD will pull just fine and the weight savings and lack of wear on 4WD components will save gas and repair money. If you get into a soft ground scenario, engage the locker and pull out. When I was wheeling, I was amazed at where I could go with a locked rear end and the front hubs unlocked. Biggest advantage of the 4WD at this point would be the low range. A lot of extra expense for an occasionally used advantage. Again, if you are constantly pulling in muddy lots to ECORS events and getting stuck means a no-show at an event, by all means, get the 4WD. I just figured if I ever got to a point where I had a tow rig, if the tow rig got stuck, I'd unload the towed vehicle (a 4WD Jeep) and pull the truck to get it unstuck and load up again. I never got to that point, so I'll never know.