There are many instances where a speaker doesn't need to be a contact, and conversely isn't a Dovetail user. (Most frequently when an invited user hasn't accepted yet).
There's extensive threads in the Dovetail community slack already.
I'm starting to see occasional "junk" contacts populate because a user's name didn't pop up by the time the user hit "enter" to name a user. Conversely, users who haven't been trained yet to use the contacts database shouldn't be able to accidentally make contacts.