please please, do some research on google scholar on inbreeding and the effects on genotypes and phenotypes.
past practises of culling dogs (that were bred from wolves) have lead to breeds such as labs and bulldogs etc. labs are now prone to joint and bone disorders and well.. bulldogs need to be artificially inseminated. tiny dogs are prone to low blood sugar... the list goes on.
In order to "fix" your genetic problems - you need to introduce new blood so that there is greater genetic variation and diversity. that way more dominant traits take over the lesser uncommon traits.
with fish - most people do inbreed them; but even the most pedantic breeders will try sourcing new blood for their lines if they can.
(hypothetical) Think of a situation where someone breeds a guppy for its red eyes - finally gets a spawn of pure red eyed babies (by crossing back and what have you) and it just so happens that the gene for red eyes also means that the fish is almost completely blind. the generations prior to this would have produced some normal eyed offspring, most of which would have carried the gene for red eyes. there is a good chance that the red eyed babies from the black eyed guppies are not blind. culling the black eyed babies only mucks up your only chance of having a semi normal guppy. (the rrrr gene {red eye, eye disorder} would only be carried by the fish that are being selected for, while the RRrr (black eye, with red eye genes not expressed are fine).
now im not too concerned about guppy genetics to be honest - the odds of an aquarium strain making it in the wild are too slim - but you should not cull any thing based on a phenotypical trait, cause all you are doing then is narrowing the genetic pool, and then your strain of fish will all be susceptible to the same problems, so 1 infection could ruin all your years of hard work.