By showing without having his hand on his weapon he MIGHT be revealing that he does not have the intent to shoot you if you give him what he wants. But maybe he is just confident in his quick draw and wants you to hand over your wallet before he shoots and makes a quick get away.
A deadly threat has clearly been made but drawing and defending yourself may not be the best option since it could easily escalate a situation that could be deescalated in other ways. This is why I carry a money clip with a little cash and a CC card in one pocket and my wallet in another. I would likely talk calmly to him while pulling out the money clip and tossing it at him. While at the same time initiating my next action which would most likely be running away, but could be diving for cover and/or drawing and maybe shooting depending on my read of the criminals intent and the lay of the land around me.