@stickbender , that's all about genetics.
Cross breeding will not damage the plant production at all, it will only change the genetics of the produced seeds, meaning that if a hot pepper was cross pollinated with a bell pepper, the seeds will produce a different plant with a fruit that's less in heat.
If you plan on using the seeds from the peppers , don't allow cross pollination.
However I love creating new types of peppers, by cross pollinating two hot peppers you will get a daughter plant with the two characteristics and different color blends.
It's beatiful to see nature developing in different ways.
That's how peppers like the Carolina Reaper and Ghost Pepper's have been created , by cross-pollinating.
Peppers are my favourite plant to grow in my backyard.