What would happen if two different species mated?

The thing that allows us to to tell two species apart is also the thing that keeps them from mating successfully, their genetics. When two members of the same species mate, the haploid cells (egg and sperm) unite to create a full diploid cell with the full amount of chromosomes. In the case for humans a full set is 23 pairs or 46 individual chromosomes.

However, when two members of different species mate their genetic information is incompatible. The number of chromosomes, among other things, prohibits viable offspring in most cases. An example of successful cross-species breeding would be the mix between a Tiger and a Lion which is a viable creature and has been created before. However, it is genetically incapable of having children of its own which is why they don't propagate.

I know this seems counterintuitive because of all the species we have on Earth but just think of it in this way. When two individuals of the same species become different enough so they can no longer breed successfully, they're considered different species and their phylogenetic trees will most likely never merge again.