First off, I assume you are wearing gloves. Harborfreight now has 5, 7 and 9 mil gloves.
I still have some 5mil ones left, but I'll probably try some 7 or 9 next, which should do an even better job insulating from the hot oil. The 5mil actually does a pretty good job already, but it rips too easily, especially when my hands are drenched in sweat this time of the year.
I prefer to let out the churned up engine oil ASAP, so any entrained contaminants have a better chance of getting flushed out with the used oil. I pretty much open the bottom (outside) drain plug right after warning up the engine. At that point, I'm more leery of the hot exhaust flanking the oil sump. I usually wear thick leather gloves to crack the drain plug loose first, then switch to the nitrile gloves to spin it out, which I can do without much oil getting on my fingers.
At that point, I just let the bottom sump drain out and switch my attention to the oil filter. I probably could do that in 2min if I rush, but I just take my time. Maybe go inside the house to cool off a bit from the hot Houston Summer heat.
By the time I am done snugging up the replacement filter, putting my oil filter wrench away, packaging up the old oil filter in a plastic bag, so on and so forth, the oil - and more importantly, the exhaust pipes - have cooled off substantially. That's when I remove the inner drain plug. The oil is still hot, but not dangerously so. The exhaust pipes are also not hot enough to burn me, if I accidentally bump my hand against it.