Why is the nitrogen's lone pair part of sp2 orbital in pyridine but part of p orbital in pyrrole?

DISCLAIMER: VERY LONG ANSWER!

As a quick review, recall that there exist three `sp^2` orbitals upon hybridization, and four `sp^3` orbitals upon hybridization.

This is inherently a difficult phenomenon to answer, so we'll have to do a deep symmetry analysis as well as considering general aspects of VSEPR theory.

Ultimately we will find that the reasons why the lone pair in pyridine is outside the ring and in pyrrole it is inside the ring are:

• Pyrrole, a five-membered ring has the same number of `pi` electrons as pyridine, a six-membered ring, which results in an extra, fourth electron group on pyrrole's `sp^2` nitrogen (yes, this is unusual).
• Pyrrole, for purposes of energetic favorability (aromaticity), has `sp^2` hybridization, despite having four electron groups, so the `2p_y` orbital stays unhybridized, allowing it to delocalize electron density throughout the ring.
• If hydrogen is to be bonded to pyrrole, then it is the `sp^2` orbital that must align towards hydrogen to overlap and form the bond. Therefore, the unhybridized `2p_y` orbital is the one that holds the lone pair.

Below, I actually go into how I know this.

BACKGROUND INFORMATION: OVERVIEW OF ORBITAL SYMMETRY AND IRREDUCIBLE REPRESENTATIONS

This is a difficult topic, but I'll try to keep it not too complicated. Pyrrole and pyridine are classified as `C_(2v)` molecules, and pertain to this character table:

![http://www.webqc.org/]()

[`A_1`, `A_2`, `B_1`, and `B_2` are called "irreducible representations". When I say "`A_1` symmetry" or "`B_2` symmetry", I am referring to these.]

As for the rest of the table, focus on the first five columns (first column contains `A_1, . . . , B_2`). Basically:

• `hatE` is an identity operation that returns the same molecule back without doing anything (defined for the purpose of completeness).
• `hatC_2(z)` is a rotation operation on the `z` axis that returns the same molecule upon rotating `\mathbf(180^@)` about the `z` axis. For both molecules, this is the only rotation axis they have.
• `hatsigma_v(xz)` is the reflection plane along the `x` and `z` axes. This operation returns the molecule reflected through the `xz` plane (coplanar with the ring).
• `hatsigma_v'(yz)` is the reflection plane along the `y` and `z` axes. This operation returns the molecule reflected through the `yz` plane (perpendicular to the ring).

The `1` in the table tells you that the operation returned the same molecule back, with orbital lobes of one sign landing on orbital lobes of the same sign.

The `-1` in the table tells you that the operation returned the same molecule back, with orbital lobes of one sign landing on orbital lobes of the opposite sign.

Example:

A `2p_y` orbital perpendicular to the ring returns:

• `1` from `hatE`
• `-1` from `C_2(z)` (since it becomes vertically flipped)
• `-1` from `sigma_v(xz)` (since it becomes vertically flipped)
• `1` from `sigma_v'(yz)` (since it becomes the same orbital again)

So, it has `\mathbf(B_2)` symmetry.

A similar analysis with the (spherical) `1s` orbital of hydrogen would give you `\mathbf(A_1)` symmetry, the totally symmetric "irreducible representation".

The rest of the answer assumes you know how to do this, so please familiarize yourself with the example above before moving on.

A rule you should keep in mind for the rest of this answer:

Two orbitals transforming under different "irreducible representations" cannot overlap and make a bond. Example: `A_1 ne B_2`, so an orbital of `A_1` symmetry is not compatible with an orbital of `B_2` symmetry.

PART 1: CONSIDERING THE NUMBER OF PI ELECTRONS: SKETCHING MO DIAGRAMS (SYMMETRY)

A quick-and-dirty way of sketching MO diagrams for these molecules is using a Frost circle mnemonic (not to be confused with a Frost diagram).

This is how it turns out for the `pi` systems of each molecule (note that pyridine diagram has the two `sp^2` valence electrons that aren't in the ring, omitted):

(pyrrole's group orbitals are here, and pyridine's group orbitals are based on benzene.)

Looking at these MO diagrams (not entirely to scale), both have six electrons in bonding "group orbitals", and...

• The energy of each "group orbitals" increases with the number of nodes (`0,1,2`) in the "group orbitals".
• The six `pi` electrons are all capably delocalized throughout the ring for both molecules, because they are all in bonding orbitals.
• None of the symmetries in the MO diagram are `A_1`!!
• Recall that the `1s` orbital of hydrogen belongs to the `A_1` "irreducible representation"!!

What we get from these Frost circles is that:

• Due to pyrrole having the same number of electrons as would be needed on the six-membered pyridine ring for aromaticity, the pyrrole nitrogen has an extra electron group.
• The `sp^2` (`A_1`) orbital that bonds with hydrogen in pyrrole cannot overlap with any `2p_y` orbital in the aromatic ring. These `sp^2` orbitals must be outside the ring.
• The `sp^2` (`A_1`) orbital that contains the lone pair in pyridine cannot overlap with any `2p_y` orbital in the aromatic ring either. These `sp^2` orbitals must be outside the ring.

PART 2: CONSIDERING THE NUMBER OF ELECTRON GROUPS (VSEPR THEORY)

Pyridine -

In pyridine, nitrogen only has three electron groups, with "true" `sp^2` hybridization (not merely forced by the ring constraints and the hope for aromaticity), so the lone pair is in the third, nonbonding `sp^2` orbital (`A_1` symmetry).

Since the `sp^2` orbital is the wrong symmetry (`A_1` symmetry), it cannot delocalize electron density into the ring, which contains `2p_y` orbitals (individually `B_2` symmetry, but `A_2` or `B_2` as a group; either way, `A_1 ne A_2 ne B_2`).

The result of this incompatibility is shown by its sticking out of the ring.

Pyrrole -

In pyrrole, nitrogen actually has four electron groups (three `sigma` bonding, one `pi`-bonding lone pair), but...

• the constraints of the ring...
• the hope for aromaticity...

...these conditions make it more energetically favorable to ideally have `sp^2`-hybridized orbitals (`A_1` symmetry) instead of `sp^3`-hybridized orbitals (also `A_1` symmetry). The third `sp^2` orbital is used to bond with hydrogen.

If pyrrole's nitrogen did ideally have `sp^3` hybridization (`A_1` symmetry), it could still bond with hydrogen, but the `sp^3` orbital would not be able to delocalize electron density into the ring because it would be the wrong symmetry to overlap with the `2p_y` orbitals (`B_2` symmetry, and `A_1 ne B_2`).

(NOTE: this image is purposefully incorrect!)

From assuming `sp^3` hybridization, we would falsely conclude that pyrrole would not be aromatic (which we know is wrong).

So, pyrrole has to be `sp^2` hybridized on its nitrogen, because:

• That keeps it aromatic.
• The third `sp^2` orbital is required to successfully bond to hydrogen.

Therefore, the fourth orbital of nitrogen (the `2p_y`) cannot hybridize unless the aromaticity is broken.

Instead, it stays unhybridized and holds the lone pair because that way, electron delocalization is allowed due to symmetry compatibility, and provides energetic stability (aromaticity).

OUR CONCLUSION

I know that was a long answer, so here's all the information gathered into a summary.

Ultimately we found that the reasons why the lone pair in pyridine is outside the ring and in pyrrole it is inside the ring are:

• Pyrrole, a five-membered ring has the same number of `pi` electrons as pyridine, a six-membered ring, which results in an extra, fourth electron group on pyrrole's `sp^2` nitrogen (yes, this is unusual).
• Pyrrole, for purposes of energetic favorability (aromaticity), has `sp^2` hybridization, despite having four electron groups, so the `2p_y` orbital stays unhybridized, allowing it to delocalize electron density throughout the ring.
• If hydrogen is to be bonded to pyrrole, then it is the `sp^2` orbital that must align towards hydrogen to overlap and form the bond. Therefore, the unhybridized `2p_y` orbital is the one that holds the lone pair.