Mike wrote:
> an the
> pixel data for whole frame from camera could be read out in lets say 2
> ms

No. Have a look at the specificatin of the camera in your link. There's 
a chapter "Data Output Format" which tells you, how fast you can get the 
data out of the camera. Some calculation will give you just the results, 
which are already given in the headlines of the camera description 
(92frames/s with LVDS-interface, 23fps with CMOS interface). I.e. these 
framerates are already limited by the interface timing of your camera.

Mike wrote:
> read out in lets say 2
> ms, buffer the frame to DDR and do processing on FPGA (we have 18 ms)?

That's wrong thinking: you can read out a frame during ~20ms, and in 
parallel you can process the preceeding frame. Things in FPGAs happen in 
parallel, you don't have to add up readout-time and processing-time.

Image sensor with FPGA. I would like to get more understanding on 
reading out the pixel data from sensor.
For example, if the sensor has 4 LVDS pairs with 720 mbps each pair. Can 
I read out the pixel data from sensor at full rate 720 mbps x 4. So if 
the pixel data for frame is 2.88 mega bit, it would take 1 ms to read 
out the frame.

Achim S. wrote:
> Mike wrote:
>> an the
>> pixel data for whole frame from camera could be read out in lets say 2
>> ms
>
> No. Have a look at the specificatin of the camera in your link. There's
> a chapter "Data Output Format" which tells you, how fast you can get the
> data out of the camera. Some calculation will give you just the results,
> which are already given in the headlines of the camera description
> (92frames/s with LVDS-interface, 23fps with CMOS interface). I.e. these
> framerates are already limited by the interface timing of your camera.
>
> Mike wrote:
>> read out in lets say 2
>> ms, buffer the frame to DDR and do processing on FPGA (we have 18 ms)?
>
> That's wrong thinking: you can read out a frame during ~20ms, and in
> parallel you can process the preceeding frame. Things in FPGAs happen in
> parallel, you don't have to add up readout-time and processing-time.



Thank you for the reply. For low latency applications, for normal 
cameras, we start processing when we acquired enough data. For example, 
Gaussian filter with 3x3 kernel required 3 lines to buffered in BRAM. If 
we are to buffer the whole frame in DDR before starting processing, the 
latency is 20 ms (for 50 fps).

So for global filter camera, the limitation still applies, is not 
possible to read the out the pixel data for whole frame in fraction of 
the frame time?

Typically the sensor spec defines the modes and options regarding 
shutter Options

Mike wrote:
> Thank you for the reply. For low latency applications, for normal
> cameras, we start processing when we acquired enough data. For example,
> Gaussian filter with 3x3 kernel required 3 lines to buffered in BRAM.

You can still do that: start processing, as soon as you have enough data 
available. I didn't want to recommend that you have to read the full 
frame before you process, that's up to you. I just wanted to remind you, 
that reading and processing can be done in parallel, the respective time 
does not simpy add up like 2ms+18ms=20ms.

Mike wrote:
> Can
> I read out the pixel data from sensor at full rate 720 mbps x 4. So if
> the pixel data for frame is 2.88 mega bit, it would take 1 ms to read
> out the frame.

I read the following numbers:
620Mbit/s x 4 in 8b/10b coding (i.e. ~62MByte/s * 4 ignoring any 
overhead)
And a full frame has 2.3MPixel, which is much more data than 2,88 megat 
bit.

If you use monochrome operation with 8bit per Pixel, you end roughly at 
2,3MByte/(62MByte/s x 4)= 9,3ms (best case estimation). This corresponds 
to the 92fps you can get out the camera if you need the full frame.