What does a closed shell mean?

It means there are no unpaired electrons. So the only filled orbitals are:

#ul(uarr darr)#

On the other hand, open-shell species have unpaired electrons in their ground-state electron configuration, i.e. something like:

#underbrace(ul(uarr color(white)(darr))" "ul(uarr color(white)(darr))" "ul(uarr color(white)(darr)))_(2p)#

In group theory,

• closed-shell atoms are said to have a #""^(1) "S"_0# (singlet-S-zero) state.
• closed-shell molecules have a #""^(1) Sigma_g^+# (singlet-sigma-gerade plus) state.

That's just another way to say that there are zero unpaired electrons in all occupied orbitals.

Atoms like #"Kr"# (noble gas), #"Zn"^(2+)# (argon configuration), etc. are closed-shell.

#"O"_2^(2-)# is an example of a closed-shell molecule, whereas #"O"_2# is an open-shell molecule.

Various open-shell cases exist. For instance... only a few homonuclear diatomics are open-shell:

• #"O"_2# ground state: #""^(3)Sigma_g^(-)#, open shell

#underbrace(ul(uarr color(white)(darr))" "ul(uarr color(white)(darr)))_(1pi_g^"*", " "2pi_g^"*")#

• #"O"_2^(+)# ground state: #""^2 Pi_g#, open shell

#underbrace(ul(uarr color(white)(darr))" "ul(color(white)(uarr darr)))_(1pi_g^"*", " "2pi_g^"*")#

• #"B"_2^(+)# ground state: #""^2 Pi_u#, open shell

#underbrace(ul(uarr color(white)(darr))" "ul(color(white)(uarr darr)))_(1pi_u, " "2pi_u)#