Those white things are cocoons. When they hatch the young will start to eat the caterpillar alive. That caterpillar is a very destructive guy. He might just eat your pepper plant before he is eaten. I suppose you could remove the cocoons and save his miserable life but why?
Actually, the caterpillar-eating happens
before the cocoons. That Tobacco Hornworm (
Manduca sexta) pictured above is already gutted inside and as good-as-dead. First a gravid female wasp (
Cotesia congregata) injects her eggs inside the caterpillar with her ovipositor/stinger. Next the eggs hatch into wasp larva that feed and fatally damage the hornworm in the process. Then the larvae emerge from the dying caterpillar, pupate and finally eclose as adults, to fly away, mate, and begin the cycle again. I do hope Midwestnoob left the cocoons on the caterpillar, because the wasps are valuable bio-control and the hornworm was a goner anyway.
This is the finest ideal of organic gardening, with predators, parasitoids, and parasites doing the pest control for us.... and once toxic pesticides are use this beautiful system is destroyed because the pesticides kill the predators, parasitoids, and parasites worse than they do the pests and then its a long road back to restore the ecosystem. Poisons are not the way.
I also want to give the Tobacco Hornworm (
Manduca sexta) its due as well. It, and the closely related Tomato Hornworm (
Manduca quinquemaculata) are significan pests on many crops in the Nightshade Family (Solanaceae). However, they also grow up to become beautiful Sphinx Moths (Hawk Moths) that are important pollinators for many native plants. We shouldn't want to eradicate them, we just need to control their numbers in crop situations.
Below is a very educational video that explains so much in a very entertaining way.