gcc -g vs not -g and strip vs not strip, performance and memory usage?

If binary file size is not an issue, are there any drawbacks using -g and not strip binaries that are to be run in a performance critical environment? I have a lot of disk space but the binary is cpu intensive and uses a lot of memory. The binary is loaded once and is alive for several hours.

EDIT:

The reason why I want to use binaries with debugging information is to generate useful core dumps in case of segmentation faults.