Memory map -> large elf file

When I link my project a very large elf file is created. Here is my
memory map:

  MEMORY
  {
  FLASH      : ORIGIN = 0x00000000, LENGTH = 0x000FC000
  CODE      : ORIGIN = 0x000FC000, LENGTH =  0x00004000
  CODE2      : ORIGIN = 0x00100000, LENGTH =  0x04000000
  MEM      : ORIGIN = 0x04100000, LENGTH = 0xA3F00000
  MVL      : ORIGIN = 0x80000000, LENGTH = 0x00100000
  QLOG       : ORIGIN = 0x80100000, LENGTH = 0x10000000
  HDWR      : ORIGIN = 0xFE800000, LENGTH = 0x017FFFFF
  }

Basicly it is just filling in the gaps in memory with zeros, which
results in a huge elf and bin file.  Is there a link option that will
keep it from doing this?

Thanks,
Dustin