You get chicken skin (which is officially called keratosis pilaris ) when dry skin doesn't slough off like it should and it clogs your pores. Your best bet is to exfoliate, either physically with a scrub or chemically with alpha hydroxy acids, and moisturize.
DermaDoctor makes a line specifically for "chicken skin" or keratosis pilaris -- there is a treatment called KP Duty (http://www.sephora.com/browse/product.jhtml?id=P73
I read any article a few months ago by Dr. Jessica Wu and she suggested using Bliss FatGirlScrub to get rid of chicken skin. My daughter and husband both have it so I bought the product for them and it is working very well.
First Aid Beauty Ultra Repair Cream is a thick, rich, emollient product that hydrates deep down with exceptional penetration. This cream is for anyone with severely dry, scaly skin due to harsh winter weather, aggressive cosmetic treatments, or any of the following conditions: atopic dermatitis, irritant eczema, allergic eczema, and keratosis pilaris. It will help with itching and irritation and you can find it at: http://sephora.com/browse/product.jhtml?id=P248407
If your condition continues or worsens, I would recommend going to see a dermatologist.
I hope this helps you.
I know this answer is coming to you quite late but try fresh aloe vera. Cut a three inch section off the plant and then cut in half length wise. Rub it on your arm at night before bed and you will see a huge difference in the morning.