I opened the emails, read them all and then deleted them off the server
from my computers mail client. The icon remained even after doing
another get mail and being told that there was no mail. I sent a test
message and this time told the phone to delete it off the phone and
server. Not long after the icon disappeared.

Its either the latter that did the trick or the notifier is really
sluggish to pick up that there are no longer any mails.

No its definitely the icon for emails. Its operation just seems a
little disjointed from the actual status of the email inbox.