Tooth enamel, or hydroxyapatite, has a molar mass of 422 g/mol. Its composition is 28.5% calcium 22% phosphorus, 49.3% oxygen, and 0.2% hydrogen. What is the molecular formula?

I believe that something is not right with the values given to you. I will let you know why I say that as I go through with the solution.

Your goal when trying to determine the molecular formula of a chemical compound is to determine exactly how many atoms of each constituent element you get per molecule (or formula unit) of your compound.

Since the problem provides you with the compound's molar mass, you can use the given percent composition to determine how many moles of each element you get per mole of the compound.

In this case, hydroxylapatite is said to have a molar mass of #"422 g mol"^(-1)#. This tells you that one mole of hydroxylapatite has a mass of #"422 g"#.

Take a #"422-g"# sample of hydroxylapatite and use the compound's percent composition to determine how many grams of each element it contains

#"For Ca: " 422 color(red)(cancel(color(black)("g compound"))) * "28.5 g Ca"/(100color(red)(cancel(color(black)("g compound")))) = "120.27 g Ca"#

#"For P: " 422 color(red)(cancel(color(black)("g compound"))) * "22 g P"/(100color(red)(cancel(color(black)("g compound")))) = "92.84 g P"#

#"For O: " 422 color(red)(cancel(color(black)("g compound"))) * "49.3 g O"/(100color(red)(cancel(color(black)("g compound")))) = "208.05 g O"#

#"For H: " 422 color(red)(cancel(color(black)("g compound"))) * "0.2 g H"/(100color(red)(cancel(color(black)("g compound")))) = "0.844 g H"#

Now use each element's molar mass to determine how many moles* of each you get in that sample

#"For Ca: " 120.27color(red)(cancel(color(black)("g"))) * "1 mole Ca"/(40.076 color(red)(cancel(color(black)("g")))) = 3.001 ~~ "3 moles Ca"#

#"For P: " 92.84color(red)(cancel(color(black)("g"))) * "1 mole P"/(30.974color(red)(cancel(color(black)("g")))) = 2.997 ~~ "3 moles P"#

#"For O: " 208.05 color(red)(cancel(color(black)("g"))) * "1 mole O"/(15.9994color(red)(cancel(color(black)("g")))) = 13.004 ~~ "13 moles O"#

#"For H: " 0.844color(red)(cancel(color(black)("g"))) * "1 mole H"/(1.00794color(red)(cancel(color(black)("g")))) = 0.837 color(red)(~~1)#

Now, this approximation

#0.837 ~~ 1#

is not as clean as the others. However, you must use it because you're dealing with the mass of one mole of the compound, which as you know must contain whole numbers of moles of each constituent element.

Think of it like this - just like one formula unit of the compound cannot have fractions of atoms, one mole of the compound cannot have fractions of moles.

So, the molecular formula of hydroxylapatite comes out to be

#color(green)("Ca"_3"P"_3"O"_13"H")#

This is equivalent to

#color(green)("Ca"_3("PO"_4)_3("OH"))#

The actual molecular formula of hydroxylapatite is #"Ca"_5("PO"_4)_3("OH")#, so something is definitely off with the values given to you.