A search for decays of the Higgs boson into a pair of new spin-zero particles, H -> aa, where the a-bosons decay into a b-quark pair and a muon pair, is presented. The search uses 36.1 fb(-1) of proton-proton collision data at root s = 13 TeV recorded by the ATLAS experiment at the LHC in 2015 and 2016. No significant deviation from the Standard Model prediction is observed. Upper limits at 95% confidence level are placed on the branching ratio (sigma(H)/sigma(SM)) x B(H -> aa -> bb mu mu), ranging from 1.2 x 10(-4) to 8.4 x 10(-4) in the a-boson mass range of 20-60 GeV. Model-independent limits are set on the visible production cross-section times the branching ratio to the bb mu mu final state for new physics, sigma(vis)(X) x B(X -> bb mu mu), ranging from 0.1 fb to 0.73 fb for m(mu mu) between 18 and 62 GeV. (C) 2018 The Author. Published by Elsevier B.V.