It is unknown whether age-related site-specific muscle loss is associated with areal bone mineral density (aBMD) in older adults. To examine the relationships between aBMD and whole-body muscle thickness distribution, 97 healthy adults (46 women and 51 men) aged 50–78 years volunteered. Total and appendicular lean soft tissue mass, aBMD of the lumbar spine (LS-aBMD) and femoral neck (FN-aBMD) were determined using dual-energy X-ray absorptiometry. Muscle thickness (MT) was measured by ultrasound at nine sites of the body (forearm, upper arm, trunk, upper leg, and lower leg). Relationships of each co-variate with aBMD were tested partialling out the effect of age. aBMD was not correlated with either MT of the trunk or anterior lower leg in either sex. In men, significant and relatively strong correlations were observed between anterior and posterior upper arms, posterior lower leg, and anterior upper leg MT and LS-aBMD or FN-aBMD. In women, significant correlations were observed between anterior and posterior upper legs, posterior lower leg, and anterior upper arm MT and FN-aBMD. LS-aBMD was only correlated with forearm and posterior upper leg MT in women. In conclusion, the site-specific association of MT and aBMD differs between sexes and may be associated with the participants’ daily physical activity profile.