You do lose some Z travel when using most adapters. You might consider a "z-setter", which determines where Z=0 is for any given bit in any given holder. Most CNC softs include the ability to auto-Zero a cutter. About the only thing they don't work well for is Forstner drills with pilots. Even then, you can make a tiny hole in the Z-setter's 'shoe' to accommodate the spur, so long as its diameter is smaller than the smallest other cutter you use.
My Forstners also have 'flush-cutters', so they're not managed by a tiny hole. It takes a hole almost the diameter of the drill. So I made two 'adapter' shoes for my Z-setter, so most cutters can use the flat adapter, and Forstners can use the one with a cavity in the middle. I use almost entirely one diameter of Forstner drill for the contract work I do. If you have multiple sizes to deal with, it might involve having more than one 'holed' adapter for the Z-setter; but that's a minor thing.
Lloyd