The ABI defines how user code-calls (library-)functions. There are three 
methods:

(1) user-code calls by using the soft-abi, library's interface is 
soft-abi and the library uses FPU emulation

(2) user-code calls by using the soft-abi, library's interface is 
soft-abi and the library uses hardware-FPU (for example v*-commands on 
Cortex-M4)

(3) user-code calls by using hard-abi, library's interface is hard-abi 
and the library uses hardware-FPU

Libraries are for example the libraries which come with the toolchain 
(like libgcc). If libraries for all options are available and all code 
of the user-application can be recompiled the 3rd method may have the 
advantage that there is no need to transfer registers-content. I did not 
look too close into the generated assembly-code to give information 
about the "overhead", maybe somebody else can.

As far as I can see (I just made a few tests with STM32F4 Cortex M4 and 
GNU tools) the main advantage of the soft-abi is the possiblity to "mix" 
libraries which are only available as object code (archives/"a-files"). 
As long as the library has been compiled with soft-abi and it uses 
FPU-emulation internally it can be linked with every other object-code 
that uses the soft-abi and everything should work. For hard-ABI all 
object-code in the application has to be compiled with this setting and 
the hardware-FPU has be be enabled. If a library uses the soft-ABI but 
the hardware-FPU interally the interface is compatible with other code 
but if the FPU is not enabled there will be core-exceptions.

You may read the getting-started.pdf for Codebench lite from 
Mentor/Codesourcery for some more information and the GNU manuals for 
even more.

Hope this helps at least it little bit.