It is my opinion you are setting yourself up for needless detainment by law enforcement. There are a number of excellent concealment mag carriers on the market which allow for concealment comfortably. Even minimal displaying can cause concern by the general public.
I’m also a concealed carrier in CA and I am not aware of any law you would be violating. As far as a permit violation, that could depend on the conditions set forth by the Sheriff of your county who issued the permit. I’d be surprised that would be a condition though.
I agree with @Frank50, there are some great magazine holders and magazine carry options that openly carrying a spare mag is probably not the answer. What kind of attire do you usually wear? What kind of gun/magazine do you typically carry?
I use Alien Gear’s Cloak Mag Carrier in the OWB configuration under my untucked shirt.
A closed, multi tool type would be good, but I haven’t found any with closed top and sides that conceals well enough. Most are strap tops that show the contents too much.
Seems like this is going to work for $10, firm form, not soft sided, closed top, looks like a knife rather than a magazine, and user reviews say it holds a mag nicely.
Outdoor Edge 4.5" Multi-Use Nylon Holster with Belt Clip & Carabiner, Black (FS-45)