The added functionality proposed in this post will be included in one of the next releases of LUFA. Please always check the official release of this framework.
If you use windows you currently have to recompile the firmware (based on LUFA) to switch between the LIBUSB variant (avrdude compatible) and the ‘normal’ version (AvrStudio). I think that should be changed to a run-time option. It wastes FLASH space, but there is plenty left.
Right now I don’t really understand enough of USB to know what exactly I’ve changed (or why) in the associated configuration, but now the way of how the programmer enumerates can be changed by pressing its reset button. The ‘MCUSR’ register is queried early and the results of its evaluation are stored reset-persistent in ‘use_libusb’. I’ve added two configuration descriptors for the two cases and replaced every instance of the preprocessor checks for ‘LIBUSB_DRIVER_COMPAT’ with a check of ‘use_libusb’. As said before it is not nice and wasteful, but it works ;-)
Connect the device to your computer and it enumerates in a way that is compatible with avrdude + linux and AVR Studio + windows, press the reset button and it re-enumerates in ‘LIBUSB mode’. It will revert back to normal once you disconnect power or press the reset button again.
I hope this is useful.