Implementing a complex algorithm on FPGA

We basically do the following steps:

 - Bilateral filtering with 11 pixel window width,
 - Sift feature extraction and matching according to previous frame,
 - Finding position of camera,
 - Basic 3D estimation of features and send to the computer with image.

If we decide that going the FPGA way will improve the performance 
dramatically, we will consider hiring an enginner to do the job. So, 
what we are interested in is whether implementing such algorithms on 
FPGAs are efficient or not.

if you are using a standard camera with CameraLink or GigE interface: 
there is a smart solution from SiliconSoftware. It simplifies the FPGA 
software development by using graphical programming with function 
blocks.
http://www.silicon-software.de/en/index.html

Hi,

Let me comment:

> - Bilateral filtering with 11 pixel window width,

Can work very fast on an FPGA and is scalable (the limit being your FPGA 
resources).

> - Sift feature extraction and matching according to previous frame,

The basic feature extraction (corner detection, etc) can be put into a 
FPGA as well. However, more complex decision algorithms should probably 
run in software.

> - Finding position of camera,
> - Basic 3D estimation of features and send to the computer with image.

That sounds like a job for the software as well.

IMHO, the frame grabber or camera link based solutions are not very 
elegant solutions and graphical programming tools may lead the way to a 
prototype, but not to the real product, so eventually, you have to redo 
it all over again.

If no extreme memory bus bandwidth is required, I could recommend a 
DSP/FPGA companion (I have made good experience with Blackfin and Xilinx 
FPGA WRT to intelligent cameras). If you don't need the full frame rate 
output over ethernet, the 100M interface will do.

The embedded approach will only work well when there is a linear 
processing chain and no requirement of massive bandwidth, i.e. something 
like:

image data -> fpga preprocessing -> dsp postprocessing -> output

What you'll have to do at an early stage, is to prototype your 
"accelerated" core operations and estimate the resources before buying 
hardware.

Cheers,
- Martin

To me, the strongest advantage of an FPGA is sizeable accuracy and 
calculation precision especially for technical applications where often 
more than 32 bit are required. But you have to face the fact that 
implementation is more complex and causes much more design expertise 
then "simple" C in order to get most of of the system.

Is there anybody out there working with abstract modelling to creat 
complex algorithms in FPGAs?

Which tools do you use to implement, and which to verify?

> We assemblied a really powerful computer but our framerate is about ~15.
I think you have to find the bottleneck.
-Is it the transport from Camera to Framegrabber .
-Is it the transfer from Framegrabber to the main memory.
-the Transport from main Memory to GPU memory
-The Algorithm on GPU
-the Transport from GPU memory back to main memory
-the Transport from main memory back to GPU 8) to show it on the Screen 
;-)
How much is the Framerate in live view without GPU. Best use the 
original Software of the Framegrabber.

Uwe wrote:
> I think you have to find the bottleneck.
guess if he didn't find it during the 1.5y this thread resides, there is 
no chance he will ever make it.

As for all designs I came across so far I could state the the data 
transportation from and to the GPU over what so ever is always the 
bottle neck even if one uses pcie.

This is the reason why people commonly use FPGAs for this.

As soon as data interchange becomes more complex than just shift from 
one cell to the next (what GPU-elements could do very well) the heaven 
like power oft the GPU systems gets down to it's knees.

> We use parallel programming techniques like CUDA and it increased the
> speed significantly but we were thinking whether reimplementing this
> algorithm in FPGA would increase the speed.
This is the point, whereby "parallel techniques" does not tell the full 
truth about a design. CUDA and FPGA-DSP techniques differ significantly 
from each other and while stepping from C and DSP based processing from 
parallelly coded CUDA processing is small - transferring algorithms to 
FPGAs will fail if fundamental issues are not understood correctly.

At first glance, the FPGAs will perform lower when only fed with soft 
core like ALUs performing DSP-based processing. Totally new structures 
will have to be invented first to overcome this limits and take 
advantage from the FPGA's real capabilities.

Since the tree of information looks mostly like this:

Design - Software - DSP
                  - CUDA GPU

       - Hardware - FPGA-softcore
       -          - FPGA-calculation chains

it will never be possible transform a design from a software to FPGAs 
efficiently.