Yikes, this is why programming early in the morning is a bad idea...
I had already done everything, including a variable to choose which gamepad slot to use, and somehow got confused looking at my init code, rather than looking at my code that actually checks inputs.
Thanks though, was the kick in the brain I needed to realize I was being dumb.