East African highland bananas (Musa spp. AAA group) are an important staple in the Great Lakes region of East Africa. Their production has declined due to pests and diseases. Breeding for host plant resistance is a sustainable option for addressing this challenge. Understanding the relationships between growth parameters and bunch weight (i.e., yield) is crucial to guide breeding efforts for this crop. We investigated cause-effect relationships, through path analysis, in bunch weight of East African highland banana derived hybrids, their parents and grandparents. These family structures were planted in a 7 × 8 rectangular lattice design, replicated twice. Genetic gains for bunch weight (kg plant−1) and yield potential (t ha−1 year−1) were estimated. Significant increases of bunch weight and yield potential were noted from the landrace triploid germplasm, their derived primary tetraploid hybrids and secondary triploid bred-germplasm. Path analysis revealed that fruit length, circumference and number, number of hands and plant cycle number had a direct positive effect on the bunch weight. Days to fruit filling, days to maturity and index of non-spotted leaves had indirect effects on bunch weight. The average genetic gains for bunch weight and yield potential were 1.4% and 1.3% per year, respectively. This is the first report about genetic gains in banana breeding. Our findings may be useful for assessing progress and directing future breeding efforts in banana breeding.