I like the idea but sooo difficult! Maybe if you could have projectile-based weapons? Or do you get upgrades as you go...? Just dying instantly on the first floor doesn't really encourage progress. Feels like a system where you move and aim/shoot at the same time might've worked better here. So you can actually dodge attacks as you attack.
-cd-