Earthquake-induced building damage assessment is an indispensable prerequisite for disaster impact assessment, and the increasing availability of high-resolution Synthetic Aperture Radar (SAR) imagery has made it possible to construct damaged building inventories soon after earthquakes strike. However, the shortage of pre-seismic SAR datasets and the lack of available building footprint data pose challenges for rapid building damage assessment. Taking advantage of recent advances in machine learning algorithms, this study proposes an object-based building damage assessment methodology that uses only post-event SAR imagery. A Random Forest machine learning-based object classification, a simplified approach to the extraction of built-up areas, was developed and tested on two ALOS2/PALSAR-2 dual polarimetric SAR images acquired in affected areas soon after the 2015 Nepal earthquake. In addition, a series of texture metrics as well as the random scattering metric and reflection symmetry metric were found to significantly enhance classification accuracy. The feature selection was found to have a positive effect on overall performance. Moreover, the proposed Random Forest framework resulted in overall accuracies of 93% with a kappa coefficient of 0.885 when the object scale of 60 × 60 pixels and 15 features were adopted. A comparative experiment with the k-nearest neighbor framework demonstrated that the Random Forest framework is a significant step toward the achievement of a balanced, two-class classification.