Nah, puffers will eat anything! When I first started feeding them, I'd spear them on a skewer and move them around so they'd chase them, but it only took once or twice before they figured out that it's food and eat it.
I give them about an hour to eat whatever I feed, and if they don't eat it all they get nothing the next day. If they eat it all, I'll give them a tiny bit more. They're pretty smart and learn really quick that if they don't eat their dinner, they don't get fed tomorrow!