Convolutional sparse representations allow modeling an entire image as an alternative to the more common independent patch-based formulations. Although many approaches have been proposed to efficiently solve the convolutional dictionary learning (CDL) problem, their computational performance is constrained by the dictionary update stage. In this work, we include two improvements to existing methods (i) a dictionary update based on Accelerated Proximal Gradient (APG) approach computed in the frequency domain and (ii) a new update model reminiscent of the Block Gauss Seidel (BGS) method. Our experimental results show that both improvements provide a significant speedup with respect to the state-of-the-art methods. In addition, dictionaries learned by our proposed method yield matching performance in terms of reconstruction and sparsity metrics in a denoising task.