By using amber I put my nsec into only amber. If I want to use 5 clients today I still put my nsec into only amber.
The issue isn't putting it into wisp, it is that not using a signer means I need to put it more places. Each additional place is a new risk.