Image segmentation is an indispensable tool in computer vision applications, such as recognition, detection and tracking. In this work, we introduce a novel userassisted image segmentation technique which combines image decomposition, inner product-based similarity metric, and spectral graph theory into a concise and unified framework. First, we perform an image decomposition to split the image into texture and cartoon components. Then, an affinity graph is generated and the weights are assigned to its edges according to a gradient-based inner-product function. From the eigenstructure of the affinity graph, the image is partitioned through the spectral cut of the underlying graph. The computational effort of our framework is alleviated by an image coarsening process, which reduces the graph size considerably. Moreover, the image partitioning can be improved by interactively changing the graph weights by sketching. Finally, a coarse-to-fine interpolation is applied in order to assemble the partition back onto the original image. The efficiency of the proposed methodology is attested by comparisons with state-of-art spectral segmentation methods through a qualitative and quantitative analysis of the results. © Springer Science+Business Media, LLC 2012.