I’d suggest checking out duelist1954 on YouTube. He’s an editor for several gun mags. But his specialty is black powder guns. He goes into detail on all the different cap & ball guns and what it takes to make them run.
in the old westerns you will see the 'hero' take a shot with his revolver (don't make a difference what kind), then as he cocks it for the next shot he will raise it until the barrel is pointed straight up. THAT is the proper way to shoot a 'cap & ball' revolver as it allows the fired cap & pieces to fall out of the action.
I came to the conclusion, from my own experiences with my replicas, the Colt 1860 style revolver with open-top frame was perhaps more popular than the Remington 1858 with top strap, because of the ability to easily chuck the cap debris, and not jam up, as noted by Mike Wiskey, raising the revolver straight up, maybe even with a little shake. There certainly wasn't enough recoil to cause the revolver to tilt way up, though maybe it made grabbing the hammer for recocking a bit easier.
as others stated, mine did it also, I just raised and shook the pistol to dislodge the fired cap, it soon becomes second nature....
Try different sizes of caps, and different brands. You need one that will seat snugly against the nipple.