Any product you use that binds the ammonia will stuff up test readings.
If you had done a water change 3 days before, and use the Prime stuff, why did the ammonia "go down quite a lot". It shouldn't have been up.
What size is the tank, how long has it been running and what fish are in there?