Fuzzy logic provides a useful tool for dealing with decisions in which the phenomena are imprecise and vague. The current paper aims to construct a fuzzy SERVQUAL method for evaluating the service quality of banks in Malaysia for different market segments, namely conventional, Islamic and foreign banks. Further, a comparison of service quality gaps has been assessed with respect to demographic characteristics of the customers to answer a fundamental question "What do customers want from banking services and which elements of service quality is important for different groups of customers?" The data is collected using structured questionnaire from 766 bank customers, consisting of 465, 200 and 101 customers respectively from conventional, foreign and Islamic banks. First, the effectiveness of the Fuzzy linguistic scale and the Likert scale was compared by using the test of internal consistencies on dimensions of service quality. Further, the rigorous parametric statistical tests are carried out to check for the appropriateness of statistical analysis to be performed for the comparative analysis. Finally, the SERVQUAL gaps are analyzed and compared across 3 market segments and different groups of customers based on demographic characteristics by using non-parametric approach to derive the implications for the bank management. © IDOSI Publications, 2013.