Try playing around with different camera re-mappings. Unity's input manager often mistakenly maps camera to shoulder triggers instead of right analog stick for PS4 controllers.
As for the Switch controller problem, I use Betterjoy and follow these instructions: https://github.com/Davidobot/BetterJoy/wiki/HIDGuardian-and-Steam-(Big-Picture)