A search for vectorlike quarks is presented, which targets their decay into a Z boson and a third-generation Standard Model quark. In the case of a vectorlike quark T (B) with charge +2/3e (-1/3e), the decay searched for is T→Zt (B→Zb). Data for this analysis were taken during 2015 and 2016 with the ATLAS detector at the Large Hadron Collider and correspond to an integrated luminosity of 36.1 fb-1 of pp collisions at s=13 TeV. The final state used is characterized by the presence of b-tagged jets, as well as a Z boson with high transverse momentum, which is reconstructed from a pair of opposite-sign same-flavor leptons. Pair and single production of vectorlike quarks are both taken into account and are each searched for using optimized dileptonic exclusive and trileptonic inclusive event selections. In these selections, the high scalar sum of jet transverse momenta, the presence of high-transverse-momentum large-radius jets, as well as - in the case of the single-production selections - the presence of forward jets are used. No significant excess over the background-only hypothesis is found and exclusion limits at 95% confidence level allow masses of vectorlike quarks of mT>1030 GeV (mT>1210 GeV) and mB>1010 GeV (mB>1140 GeV) in the singlet (doublet) model. In the case of 100% branching ratio for T→Zt (B→Zb), the limits are mT>1340 GeV (mB>1220 GeV). Limits at 95% confidence level are also set on the coupling to Standard Model quarks for given vectorlike quark masses.