FPGA image fusion & stereo vision

Hi,

I would like to create an embedded multispectral stereovision system 
using 4 sensors (4k@60 fps).

I use 2 sensors for each eye (1 rgb and 1 near-infrared (nir) sensor 
with a dichroic beam splitter) in order to capture rgb & nir videos for 
each eye.

I have to fusion the videos capture by the sensors for each eye (rgb and 
nir videos). Then, i have to merge the two videos in order to display it 
on a VR headset (HDMI output).

Is this project possible only by using a single FPGA connected to 4 
sensors and a HDMI VR Headset without any CPU/GPU (Tegra) ? Which FPGA 
is the most suitable according to your experience ?

Thank you.