Silly is a town in Belgium in the Walloon region, and it would be ridiculous to confuse it with a balloon. Moreover, the local farmers plant vegetables, which is a smart thing to do and not deserving of this name. The local river is called the Sille Stream. After that the trail goes cold as the water does in winter.
The town is twinned with San Miniato in Italy where they found a truffle that weighed in at 5.56 pounds. For some reason they gave it to President Eisenhower who may have eaten it. The only other useful information we have is the last Duke was Marquis Ferdinand Silly-Octave Trazegnies and he died in 1740 after which he was quite dead.