Sam, I have seen similar symptoms many, many times in baby birds. When we x-ray them we see dozens of fractures throughout their body. I am not saying that your finch has this problem, but it is a possibility. It could also be a neurological deficit or some sort of toxin. If you have fed some of the crushed up boiled egg and shell then that will have bosted his recovery as it is chock full of fat, protein, vitamins A, B and Niacin as well as the all important calcium - it has enough nutrients to counteract any of the most common nutritional deficiencies so you shoud be ok with that. If he has fractures, it may be most humane to euthanise him since he may always be a cripple, even if he recovers fully. Babies do heal very quickly so if you are keen, keep him confined, and in a quiet place so he can sleep a lot. Feed him well on a balanced diet supplemented with the crushed up egg and keep him warm and in no time he will be healing and recovering! Keeping him in a tissue box is just fine for now as long as he isn't continually struggling.
Keep us updated on your birdy nursing and your little patient.