For existing Bayesian cross-validated measure of influence of each observation on the posterior distribution, this paper considers a generalization using the Bregman Divergence (BD). We investigate various practically useful and desirable properties of these BD based measures to demonstrate the superiority of these measures compared to existing Bayesian measures of influence and Bayesian residual based diagnostics. We provide a practical and easily comprehensible method for calibrating these BD based measures. Also, we show how to compute our BD based measure via Markov chain Monte Carlo (MCMC) samples from a single posterior based on the full data. Using a Bayesian meta-analysis of clinical trials, we illustrate how our new measures of influence of observations have more useful practical roles for data analysis than popular Bayesian residual analysis tools.