The "gunge" is what stops the filter filtering the water and stops removing particles, bacteria work just as well in a properly cleaned filter. I clean mine about once every 1-2 months to prevent build up in the sponges and wool, the way i clean my bio media is i leave it in it tray and put it in the sink, tip half a bucket of warm tank water over it to backwash all the gunk and then put it back into the filter. I've never had any issues with my tank re cycling during my 4-5 years of fish keeping.
If you wanted to get technical, the more gunk you have the less surface area you have in your filter, restricting optimal conditons for them and reducing space for more bacteria to live