Yeah, you need an HDMI switchbox, unless by some chance you have an A/V receiver that accepts multiple HDMI inputs.
A sub-$300 modern receiver by Denon, Yamaha, Pioneer, or Onkyo usually has 4 or more HDMI inputs. That is not just a fat chance, but a way to go.
While I agree with going to an AVR, don't forget you have to add in the cost of speakers as well. With sub-woofer, expect another few hundred to complete the system. But in my opinion, worth it.
Nvidia Shield, Roku Stick (3600), Ultra (4640), Premiere (3920), Insignia 720p Roku TV, Sharp 4K Roku TV, Windows 10 Pro x64 running Serviio and Plex on a wired Gigabit network.