Accessing Google Speech API / Chrome 11

265 Replies

I’ve posted an updated version of this article here, using the new full-duplex streaming API.

Just yesterday, Google pushed version 11 of their Chrome browser into beta, and along with it, one really interesting new feature- support for the HTML5 speech input API. This means that you’ll be able to talk to your computer, and Chrome will be able to interpret it. This feature has been available for awhile on Android devices, so many of you will already be used to it, and welcome the new feature.

If you’re running Chrome version 11, you can test out the new speech capabilities by going to their simple test page on the html5rocks.com site:

http://slides.html5rocks.com/#speech-input

Genius! but how does it work? I started digging around in the Chromium source code, to find out if the speech recognition is implemented as a library built into Chrome, or, if it sends the audio back to Google to process- I know I’ve seen the Sphynx libraries in the Android build, but I was sure the latter was the case- the speech recognition was really good, and that’s really hard to do without really good language models- not something you’d be able to build into a browser.

I found the files I was looking for in the chromium source repo:

http://src.chromium.org/viewvc/chrome/trunk/src/content/browser/speech/

It looks like the audio is collected from the mic, and then passed via an HTTPS POST to a Google web service, which responds with a JSON object with the results. Looking through their audio encoder code, it looks like the audio can be either FLAC or Speex– but it looks like it’s some sort of specially modified version of Speex- I’m not sure what it is, but it just didn’t look quite right.

If that’s the case, there should be no reason why I can’t just POST something to it myself?

The URL listed in speech_recognition_request.cc is:

https://www.google.com/speech-api/v1/recognize

So a quick few lines of PERL (or PHP or just use wget on the command line):

#!/usr/bin/perl

require LWP::UserAgent;

my $url = "https://www.google.com/speech-api/v1/recognize?xjerr=1&client=chromium&lang=en-US";
my $audio = "";

open(FILE, "<" . $ARGV[0]);
while(<FILE>)
{
    $audio .= $_;
}
close(FILE);

my $ua = LWP::UserAgent->new;

my $response = $ua->post($url, Content_Type => "audio/x-flac; rate=16000", Content => $audio);
if ($response->is_success)
{
    print $response->content;
}

1;

This quick PERL script uses LWP::UserAgent to POST the binary audio from my audio clip; I recorded a quick wav file, and then converted it to FLAC on the command line (see SoX for more info)

To run it, just do:

[root@prague mike]# ./speech i_like_pickles.flac

The response is pretty straight forward JSON:

{
    "status": 0,
    "id": "b3447b5d98c5653e0067f35b32c0a8ca-1",
    "hypotheses": [
    {
        "utterance": "i like pickles",
        "confidence": 0.9012539
    },
    {
        "utterance": "i like pickle"
    }]
}

I’m not sure if Google is intending this to be a public, usable web service API, but it works- and has all sorts of possibilities!

265 thoughts on “Accessing Google Speech API / Chrome 11”

  1. mike Post author

    Yeah- new versions of chrome are using a streaming service, that lets you pass chunked data longer than 40 seconds- but it’s a lot more involved than just making a single HTTP POST.

    I’m working on an example to use it directly, but haven’t finished it yet.

    Mike

  2. Ramesh

    Thanks for replying,

    Ok, I am waiting for the u r example Mike to pass chunked data longer than 40 seconds.

  3. Johannes

    Note to @pozy

    # curl -H “Content-Type: audio/x-flac; rate=16000″ “https://www.google.com/speech-api/v1/recognize?xjerr=1&client=chromium&lang=en-US” -F myfile=”@C:\input.flac” -k -o”C:\output.txt”

    Works excellent! Just some notes:

    1) copy & paste: use different quote signs

    2) make sure rate=16000 corresponds to your bitrate (audacity: before recording!)

    3) I had slightly better results recording mono

    Does anybody has anything about:

    * Done waiting for 100-continue

    I’m missing this milliseconds to wait for that…

  5. mike Post author

    Hey Manish,

    I believe this is using their new streaming server, not the single post-server. I’m working on an example to use this new service, but it’s much more complicated than the old single post system.

    Mike

  6. eman

    after the audio is collected from the mic, does it get stored in a buffer or a certain file?? and is it possible to access this file or get hold of it?

  7. Parikshit

    Hi guys, I am trying to run the above script with my input file but i am not able to do so…
    I converted a wav file into flac by using the online converter : http://www.convertfiles.com

    I am getting the response as below:

    {“status”:5,”id”:”89834e12f70d8f9e4d42cbf8f524bf8b-1″,”hypotheses”:[]}

    My file is a simple hello world file.

  8. mike Post author

    Usually when I’ve seen this, it’s because the bitrate on the content-type header doesn’t match the audio.

    Mike

  9. Pingback: 使用Google语音识别引擎（Google Speech API) | 长叶子的树

  12. JJ

    Hi,

    When I tried send files with 10 seconds, the server returns error 500: Internal Server Error.

    Can I do something?

    Thanks

  14. MaxZhang

    It can not recognize digits ?
    I uploaded a file contains characters and digits, only characters got recognized. why?…….

  15. Hee-dong Yoon

    Hello 🙂

    I have a problem.In order to solve this problem, i need your help.

    The problem is follow

    I using ‘c++’ and make same ‘pear’ in exercise but Result is follow.

    {“status”;5,”id”:”a8be65bfad9baf7f1717f3de46e3926e-1”,”hypotheses”:[]}

    Dictation result is nothing and Status value is 5.

    I want to know Status what it means. and I hope to know reference which described status (url)

    Thank you for read

  17. Paul

    Hi Mike,

    Any luck with the full-duplex api? I’m receiving nothing but Error 403’s…

    Hope to hear from ya,
    Paul

  18. suryateja

    thanks for the post…I’ve been looking for it…But does anyone know how to make a python script out of your ruby code??

  19. Vijay

    Anybody, please provide the wav format audio file which is have just hello voice. So I can test my application.

  20. mike Post author

    Hey Everyone,

    I haven’t had a chance to dig into the full duplex example again- swamped at the “real” job 😉

    I’m hoping to have some more time later this week.

    Mike

  26. robert rowntree

    rob@ beacon$ curl “https://www.google.com/speech-api/full-duplex/v1/down?pair=12345678901234567” & curl -X POST “https://www.google.com/speech-api/full-duplex/v1/up?lang=en-US&lm=dictation&client=chromium&pair=12345678901234567&key=…..MU” –header “Transfer-Encoding: chunked” –header “Content-Type: audio/x-flac; rate=22050” –data-binary @11.rec

    {“result”:[]}
    rob@ beacon$ {“result”:[{“alternative”:[{“transcript”:”hi how are you we have to go down to the store and see if we can get the groceries for this week so we can bring them back in the car”,”confidence”:0.971865}],”final”:true}],”result_index”:0}

    duplex session on curl works fine

  27. Anon

    Any way to take advantage of this sans-API key? I think there’s a 50 per day limit to the key, so useless for any applications other than testing.

  28. ildar

    robert rowntree: thanks for the duplex example. But I’ve got the error: Error 411 (Length Required)!!
    POST requests require a Content-length header. That’s all we know.

    Johannes’ example doesn’t work too, giving “status”;5

    🙁

  29. Mukund Kumar

    I have done the same thing but using python…but when i pass the flac file the response generated contains empty hypothesis like this
    {“status”:5,”id”:”4322b1f93df3c60bbd9fd3d5c7a22eca-1″,”hypotheses”:[]}
    Wat is the problem??any suggestion is welcomed….
    Thanks in Advance

  30. Pingback: Google Speech API – Full Duplex PHP Version | mike pultz

  35. robert rowntree

    https://gist.github.com/rowntreerob/7de7a9761edf980347b1

    Android sample above – the relevant parts that mimic exactly the Curl example. Android test OK with either amr-nb @8000 or with FLAC @22050. For audio streams of duration > 15 seconds , there are additional issues over rate of delivery of successive audio chunks. Note that if you are mapping bytes from a bigger audio file into memory and then feeding that block to the http IO channel on the UP channel , you may have to address a rough rate of delivery that mimics the rate of bits resulting from your choice of Encoding/bitRate for the audio stream. I did that by introducing and intermittently sleeping thread that writes about 2K at a time to the channel on the UP connection before sleeping for an appropriate, rough interval.

  37. Danny

    @robert rowntree
    I have been trying to build the Android Sample you posted. However, what are the values for the variables below?
    API_DOWN_URL
    PAIR
    API_UP_URL_p1
    API_UP_URL_p2
    MIN
    MAX

  38. robert rowntree

    @Danny

    private static final long MIN = 10000000;
    private static final long MAX = 900000009999999L;
    PAIR = MIN + (long)(Math.random() * ((MAX – MIN) + 1L));
    DOWN = root + dwn

    UP = root + up_p1 + PAIR + up_p2 + api_key

    https://www.google.com/speech-api/full-duplex/v1/
    down?maxresults=1&pair=
    up?lang=en-US&lm=dictation&client=chromium&pair=
    &key=
    get a key from google

  39. robert rowntree

    private static final long MIN = 10000000;
    private static final long MAX = 900000009999999L;
    PAIR = MIN + (long)(Math.random() * ((MAX – MIN) + 1L));
    DOWN = root + dwn

    UP = root + up_p1 + PAIR + up_p2 + api_key

    private static final long MIN = 10000000;
    private static final long MAX = 900000009999999L;
    PAIR = MIN + (long)(Math.random() * ((MAX – MIN) + 1L));
    DOWN = root + dwn

    UP = root + up_p1 + PAIR + up_p2 + api_key

    https://www.google.com/speech-api/full-duplex/v1/
    down?maxresults=1&pair=
    up?lang=en-US&lm=dictation&client=chromium&pair=
    &key=
    get a key from google

  42. Daria

    I try to recognize a little phraze – less than 4 seconds. tmp.flac is 16bit, 16kHz, mono. I send POST-request but have no response. No response at all. Curlib ouput “Empty reply from server”. This is my code:

    FILE *f;
    f = fopen(“tmp.flac”, “r”);
    if (f == NULL) {
    perror(NULL);
    }

    long uploaded_len = 0;
    std::string filecontent = “”;
    if (f )
    {
    fseek ( f, 0, SEEK_END );
    uploaded_len = ftell( f );
    fclose (f);
    f = fopen(“tmp.flac”, “rb”);
    char c = 0;
    for(int i = 0; i < uploaded_len; ++i) {
    fread( &c, sizeof(char), 1, f);
    filecontent += c;
    }
    fclose( f );
    }

    _curl = curl_easy_init();
    CURLcode res;

    if( _curl && f ) {
    curl_easy_setopt(_curl, CURLOPT_ERRORBUFFER, errorBuffer);
    curl_easy_setopt(_curl, CURLOPT_URL, "https://www.google.com/speech-api/v1/recognize?lang=ru-RU&xjerr=1&client=chromium&quot;);
    curl_easy_setopt(_curl, CURLOPT_VERBOSE, 1);
    curl_easy_setopt(_curl, CURLOPT_POST, 1);
    curl_easy_setopt(_curl, CURLOPT_POSTFIELDS, filecontent.c_str());

    struct curl_slist *url_list = 0;
    url_list = curl_slist_append(url_list, "Content-type: audio/x-flac; rate=16000");

    char lng[256];
    sprintf(lng, "%i", uploaded_len);
    std::string tmpstr = "Content-length: " + std::string(lng);
    url_list = curl_slist_append(url_list, tmpstr.c_str());

    curl_easy_setopt(_curl, CURLOPT_POSTFIELDS, filecontent.c_str());
    curl_easy_setopt(_curl, CURLOPT_SSL_VERIFYPEER, false);
    curl_easy_setopt(_curl, CURLOPT_HEADER, 1);
    curl_easy_setopt(_curl, CURLOPT_HTTPHEADER, url_list);
    curl_easy_setopt(_curl, CURLOPT_WRITEFUNCTION, writer);
    curl_easy_setopt(_curl, CURLOPT_WRITEDATA, &buffer);
    res = curl_easy_perform(_curl);

    if (res == CURLE_OK) {
    std::cout << buffer << "\n";
    }
    else {
    std::cout << "Error! " << errorBuffer << std::endl;
    }

    curl_slist_free_all(url_list );
    curl_easy_cleanup(_curl);
    }

    What's wrong? Why I have empty reply from server?

  43. Daria

    I found that I was doing wrong. I missed the following string:
    curl_easy_setopt(_curl, CURLOPT_POSTFIELDSIZE, uploaded_len);
    The request should contain information about file size, but mentioning it in headers doesn’t work.

  44. Chang

    i simulated your perl code, but met this : HTTP::Response=HASH(0x3e8c550)
    what’s this ? could you please help ?
    by the way I’m China(mainland)
    thanks a lot !

  45. mike Post author

    Hey Chang,

    I’m not sure what that is- usually when it says that, it’s a PERL HASH object- you could try taking it and iterating through it like a HASH and see what is looks like? or use Data::Dumper to print it out directly.

    Mike

  46. irux

    Hi , i try to do that with curl , but when the server respons me i recive words that i didnt say . I recorded the audio very well and it doesnt work.

  47. Nissim

    Hello Speech recognition Expert ,
    As hobby, I like to run Google Speech recognition on my small Linux box. The audio comes from USB audio Microphone as 16Khz 16Bit.
    It can be flac file compressed, I presume.
    Do you have experience of that ?
    If yes, can I hire someone for this help ?
    Please call my Skype: nissm.test
    Thank you
    Nissim.zur@gmail.com

  49. Pingback: 使用Google语音识别引擎（Google Speech API） |

  50. Shobhit

    It works really well with very high accuracy almost 100%. But it is too slow. It takes lot of time to get a reply from the server. I used wget to fire the request. Has anyone else experienced the same?

Leave a Reply

Your email address will not be published. Required fields are marked *