Finding Homophones of recognition

Tom D 40 Reputation points
2025-10-02T12:51:20.8866667+00:00

Hi,

We're using the azure javascript speech sdk for speech to text in language learning.

Users get a cue and have to utter the correct translation: e.g. Choose -> Kies
However sometimes (even with the prhaselist) enabled a Homophone is recognized. In this example quiche. Quiche sounds very similar except for the che sounds like sjjj and 's' in kies is a normal s.

When quiche is recognized, kies is not in the Nbest list and since the spelling is quite different we cannot match it to kies.

We tried using the pronunciation assessment with a referenceText, but when we enable this the recognizer will not return obviously incorrect answers but will return a NoMatch

Is there a way of getting homophones of a recognized word/text? Or is they are way to apply pronunciationassessment with a referenceText only after the initial recognition so we can check how close the pronuncation of 'quiche' was to 'kies'?

Azure Speech in Foundry Tools
0 comments

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Deleted

    This answer has been deleted due to a violation of our Code of Conduct. The answer was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.

    3 deleted comments

    Comments have been turned off. Learn more

  2. Alex Burlachenko 26,135 Reputation points MVP Volunteer Moderator
    2025-10-02T13:49:21.1733333+00:00

    Tom D hello,

    you are right, the pronunciation assessment is too strict and causes a 'no match' when the word is wrong, which defeats the purpose of catching homophones.

    unfortunately, the azure speech sdk does not have a built in feature to return a list of homophones or phonetic similarities. the n best list is generated from the acoustic model's confidence, not from a phonetic dictionary.

    instead of relying on the speech service for this, you could build a small phonetic matching module in your own application code.

    after you get the primary recognition result, like 'quiche', you can use a phonetic algorithm to compare it to your expected word 'kies'. algorithms like soundex or metaphone are designed to group words that sound similar. you can calculate the phonetic code for both 'quiche' and 'kies'. if the codes are similar, you can then present a hint to the user like "did you mean 'kies'?".

    another approach is to run two recognition passes. first, do a normal recognition without pronunciation assessment to get the raw text, even if it's 'quiche'. then, take that audio and run it through the pronunciation assessment with the reference text set to the correct word, 'kies'. this might give you a score showing how close the pronunciation was to the target, even if the recognized word was wrong.

    this kind of post processing is common in language learning applications where you need to handle learner errors gracefully.

    the service won't give you homophones, but you can use a phonetic algorithm in your own code to compare the recognized word to the expected word and see if they sound similar.

    regards,

    Alex

    and "yes" if you would follow me at Q&A - personaly thx.
P.S. If my answer help to you, please Accept my answer

    https://ctrlaltdel.blog/

    Was this answer helpful?

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.