The Demo Project was not accurate enough. Since the transcript obtained from google-speech-to-text is used directly as the search query for bani-db. Most of the times spellings did not match with gurbani database. So our mentor suggested us following changes in the project.
Send request to Banidb using 1st search type. (Rather than searching entire word, it should search only the first letter of each word)
Any kind of Visual cue when the mic is recording audio.
Rather than just bani name, it should display a link or entire shabad that matches the search query.
Changes Made
Backend adjustment for the first change was pretty straight forward. We thought of the long term use and adjusted the express app code just a little bit. Now the express app takes following 2 paramters in request.
Variable Name
Value
audioData
string:
Audio Data in base64 format
initials
bool:
true or false
If the "initials" value is false then the backend will return the normal transcript as it is. But if the value of "initials" variable is true then the backend will return a string that contains the first letter of each word separated with space.
First Letter Of Punjabi words: In punjabi strings, each letter and "laga'n" "matra" occupies only one index. For example if the string/word is "ਜੇਕਰ" then it's stored as [ਜ, ੇ,ਕ,ਰ]. So to obtain the first letter of each word, we simply take the first index of each string.
For the visual cues we added a glow effect around the mic component. So now when the mic is recording user's voice, it glows. Also there is a notification as pop up that let's user know that the recording has been started and recording has been stopped.
There were a ton of changes made on frontend. The biggest change was adding a dropdown menu that lets user choose from two options. Transcript or Gurbani. As explained in table above it is to obtain full transcript or only first letter of each word. If user chooses Transcript then they will get only transcript of their voice. But if they choose gurbani then the app will make request to banidb-api and find gurbani that matches with their query Code was completely refactored. Here I would like to commend Simarjot Singh for writing very clean code. He is actually very skilled in making frontend using React. Results section was also changed to match what was suggested by our mentor.
A lot of testing was done again. This time the results were much more consistent with first letter of each word search. Now we could finally move onto the next step which was the implementation in Sikhi To The Max application.