Video shot boundary detection plays an important role in video processing. It is the first step toward video-content analysis and content-based video retrieval. We develop a hierarchical approach for shot boundary detection based on the assumption that hierarchy helps to take decisions by reducing the amount of indeterminate transitions. Our method consists in first detecting abrupt transitions using a learning-based approach, then non-abrupt transitions are split into gradual transitions and normal frames. We describe in this paper, a machine learning system for shot boundary detection. The core of this system is a kernel-based SVM classifier. We present some results obtained for shot extraction TRECVID 2006 Task.