Listed Buildings - pretty much a given for all follies I have worked on, the most fun one was Rapunzel....how *do* you get a 24ft sculpture secured to a building, without leaving a single mark? (thanks again for Ben and his fabulous knowledge of ropes!)
Distance - getting things from a to b without roads leading where you want to go......the main challenge is still coming up, at Clavell Tower in Dorset. Sculpture needs to be "dismantlable" (yes, I made that word up!), so it can be taken there, on foot, in parts......although the army stretcher solution for Rapunzel's hair comes a close second - or maybe carrying a glass window, in pitch black darkness, across a large parkland (Cannon Hall).
Weather - no, not the "I hope it doesn't rain, so people come", although that's important, too. I mean the "I'm hanging a large dangle of glass from a window, if it's windy, the whole thing won't work" (Rapunzel), or indeed the "access is difficult for a few months each year, as wet weather makes the doors swell, and we can't get in" (Rotunda).
These are just some of my "fun" moments...I'll think of a few more to share!