Google Home integration (was Re: Google Home on sale)

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Google Home integration (was Re: Google Home on sale)

Eloy Paris
Hi Brian,

On Fri, Nov 25, 2016 at 12:34:15PM -0600, Brian Klier wrote:

> Eloy,
>
> I've been having some successes integrating Google Home/Assistant with
> MisterHouse, primarily due to some previous code Bruce Winter used to
> integrated Android Watches/Phones to MisterHouse through Tasker.
>
> Let me know if you have any questions or concerns.  I posted to the list
> earlier today with a fuzzy logic problem I've been encountering with this
> integration but exact matches have been working perfectly.  Very cool stuff
> indeed!

Thanks for the insight.

I looked at your message to the mailing list (the one about the fuzzy
logic problem you are having) and at Bruce Winter's blog post on how
he integrated Android Watches/Phones to MisterHouse through Tasker,
and took some time to understand how you are integrating Google Home
with MisterHouse via IFTTT. Seems pretty straightforward, and I will be
testing your same approach as soon as I take the Google Home I purchased
out of the box.

Regarding Google Home integration via IFTTT -- I am concerned about
exposing the MH web server to the Internet so the IFTTT Maker channel
can send voice commands to MH. Not only is it bad enough to expose
the MH web server to the Internet, but there isn't any type of
authentication in the /bin/runit.pl script that IFTTT talks to.
Hopefully the standard password_allow_clients mechanism applies and I'll
be able to at least specify a range of IPs from where the IFTTT request
is allowed to come from (if not I can accomplish something similar with
a firewall rule), but it still makes me very uncomfortable, plus the IP
range from where the IFTTT request will come from is not documented as
far as I can tell.

In the end, I'd prefer a solution that does not involve an action (like
turning a light on or off) being initiated by a cloud service (like
IFTTT) but right now I don't see any other way. My hope is that if/when
Google decides to publish some type of API (perhaps "Direct Actions"
as mentioned at https://developers.google.com/actions/, once that is
available), we will be able to do something more "elegant" (though I am
not sure we'll be able to completely take the cloud out of the picture,
but we'll see).

But I think that at least for now this Google Home integration via IFTTT
is good enough.

Cheers,

Eloy Paris.-


------------------------------------------------------------------------------
________________________________________________________
To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Google Home integration (was Re: Google Home on sale)

Eloy Paris
On Thu, Dec 01, 2016 at 09:53:26AM -0500, Eloy Paris wrote:

[...]

> Regarding Google Home integration via IFTTT -- I am concerned about
> exposing the MH web server to the Internet so the IFTTT Maker channel
> can send voice commands to MH. Not only is it bad enough to expose
> the MH web server to the Internet, but there isn't any type of
> authentication in the /bin/runit.pl script that IFTTT talks to.

Oh, and there is no HTTPS involved either. Perhaps the solution here is
to make Apache (or your web server of choice) do HTTPS proxying for
MisterHouse.

Cheers,

Eloy Paris.-


------------------------------------------------------------------------------
________________________________________________________
To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Google Home integration (was Re: Google Home on sale)

Brian Klier
In reply to this post by Eloy Paris
Eloy,

Great commentary!

I can add that the standard password_allow_clients mechanism DOES apply for the bin/runit.pl script, but unfortunately, I haven't found a specific IP address range that incorporates all of IFTTT yet.  Clearly, IFTTT is not using one specific IP address for these requests, and the IP address seems to switch during each HTTP GET to MisterHouse.  One could of course rename the bin/runit.pl script to something else, maybe with a long random number string as part of the file name.  This would keep the nice guys nice, but wouldn't help if the URL itself is being sniffed out.  I guess at this point in time, if someone wants to turn on my Christmas lights, then I'll deal with that when it happens!

It's fun to have the stepping stones of Google Home<>MisterHouse integration underway, security implications aside.  Certainly, I'm hoping there will be enough developers willing to help develop built-in support for Google Home once Google releases the API this month.  I believe as more people integrate these voice assistant technologies in their home, we are going to want to focus some development effort on MisterHouse's "fuzzy logic", as it is far from perfect at this point.  Voice control truly is the crown jewel of home automation control, and the future is nearly here!

Brian

On Thu, Dec 1, 2016 at 8:53 AM, Eloy Paris <[hidden email]> wrote:
Hi Brian,

On Fri, Nov 25, 2016 at 12:34:15PM -0600, Brian Klier wrote:

> Eloy,
>
> I've been having some successes integrating Google Home/Assistant with
> MisterHouse, primarily due to some previous code Bruce Winter used to
> integrated Android Watches/Phones to MisterHouse through Tasker.
>
> Let me know if you have any questions or concerns.  I posted to the list
> earlier today with a fuzzy logic problem I've been encountering with this
> integration but exact matches have been working perfectly.  Very cool stuff
> indeed!

Thanks for the insight.

I looked at your message to the mailing list (the one about the fuzzy
logic problem you are having) and at Bruce Winter's blog post on how
he integrated Android Watches/Phones to MisterHouse through Tasker,
and took some time to understand how you are integrating Google Home
with MisterHouse via IFTTT. Seems pretty straightforward, and I will be
testing your same approach as soon as I take the Google Home I purchased
out of the box.

Regarding Google Home integration via IFTTT -- I am concerned about
exposing the MH web server to the Internet so the IFTTT Maker channel
can send voice commands to MH. Not only is it bad enough to expose
the MH web server to the Internet, but there isn't any type of
authentication in the /bin/runit.pl script that IFTTT talks to.
Hopefully the standard password_allow_clients mechanism applies and I'll
be able to at least specify a range of IPs from where the IFTTT request
is allowed to come from (if not I can accomplish something similar with
a firewall rule), but it still makes me very uncomfortable, plus the IP
range from where the IFTTT request will come from is not documented as
far as I can tell.

In the end, I'd prefer a solution that does not involve an action (like
turning a light on or off) being initiated by a cloud service (like
IFTTT) but right now I don't see any other way. My hope is that if/when
Google decides to publish some type of API (perhaps "Direct Actions"
as mentioned at https://developers.google.com/actions/, once that is
available), we will be able to do something more "elegant" (though I am
not sure we'll be able to completely take the cloud out of the picture,
but we'll see).

But I think that at least for now this Google Home integration via IFTTT
is good enough.

Cheers,

Eloy Paris.-



------------------------------------------------------------------------------

________________________________________________________
To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Google Home integration (was Re: Google Home on sale)

Eloy Paris
Hi Brian,

On Thu, Dec 01, 2016 at 09:58:31AM -0600, Brian Klier wrote:

> Eloy,
>
> Great commentary!
>
> I can add that the standard password_allow_clients mechanism DOES apply for
> the bin/runit.pl script, but unfortunately, I haven't found a specific IP
> address range that incorporates all of IFTTT yet.  Clearly, IFTTT is not
> using one specific IP address for these requests, and the IP address seems
> to switch during each HTTP GET to MisterHouse.  One could of course rename
> the bin/runit.pl script to something else, maybe with a long random number
> string as part of the file name.  This would keep the nice guys nice, but
> wouldn't help if the URL itself is being sniffed out.  I guess at this
> point in time, if someone wants to turn on my Christmas lights, then I'll
> deal with that when it happens!

It looks like IFTTT uses Amazon Web Services so the HTTP request indeed
comes from "random" addresses. It'd be hard to filter based on source IP
address.

I know there is a little bit of "security by obscurity" here because an
"attacker" would have to know your public IP address, but someone who
is determined could obtain that information (searching for one's name
online, in social media, etc.).

I do think the risk of someone wanting, and then able, to mess with our
MisterHouse installations via this the IFTTT interface that we have
been discussing is low. I think the risk is higher than just turning
Christmas lights on, though -- any voice command can be executed, like
"exit mister house", or "reboot the computer".

In any case, I thought a little bit about how to improve the situation
and came up with a slightly different solution that satisfies most of my
requirements. I went about it a little differently...

I already run an Apache web server that I expose to the Internet.
In addition, I like PHP as a language to handle HTTP requests. So,
instead of handling the incoming IFTTT request from the MisterHouse
web server (and exposing the MisterHouse web server to the Internet),
I wrote a small PHP script that is invoked from Apache that in the end
executes the "mhsend" utility in MisterHouse's "bin" directory. Some
enhancements/differences from the original runit.pl are:

* A simple authentication mechanism -- an "authentication" token is sent
in the request from IFTTT, which is then checked against a hardcoded
token in the PHP script.

* Using HTTP POST instead of HTTP GET -- that way the URL is kept short
and we have more flexibility in terms of passing data from IFTTT.

* A simple mechanism to filter out commands that are not allowed to run.
For example, in my case I only allow at the moment "xxxxxx light
(on|off)".

* My Apache can serve over HTTPS, and the IFTTT Maker channel supports
HTTPS, so the request comes in encrypted. Of course, it takes longer to
establish an HTTPS session so I might just go with cleartext HTTP, but I
wanted to test that HTTPS works.

* No need to touch MisterHouse at all (no runit.pl, no MisterHouse web
server); just use the existing MH voice commands.

With these enhancements I feel comfortable opening my MisterHouse to
voice commands received from IFTTT.

(I'm including the script below, in case someone else already has a web
server capable of running PHP scripts and wants to integrate IFTTT with
MisterHouse. Instructions for the IFTTT recipe action are in the script
itself. One future enhancement is to do fuzzy command matching, BTW.)

> It's fun to have the stepping stones of Google Home<>MisterHouse
> integration underway, security implications aside. Certainly, I'm
> hoping there will be enough developers willing to help develop
> built-in support for Google Home once Google releases the API this
> month. I believe as more people integrate these voice assistant
> technologies in their home, we are going to want to focus some
> development effort on MisterHouse's "fuzzy logic", as it is far from
> perfect at this point. Voice control truly is the crown jewel of home
> automation control, and the future is nearly here!

In terms of the Google Home -- I took it out of the box today and I
have to say that I'm impressed: It became an immediate sensation in
my household and it is not hard to see the immense potential that it
has. It's so cool to be able to do so many things by just talking to it
(adding items to the grocery shopping list, asking for music to play,
asking for so many things like weather forecast, time, appointments,
etc., and now also to send voice commands to MisterHouse).

As you say, let's wait to see what can be done with the API once it is
released but in the meantime I think that what we have via IFTTT is
pretty decent. Pity the request comes from the "cloud" (which makes
things a bit slow), but I can live with that.

Voice control is definitely the crown jewel of home automation (I call
it the 'Holy Grail' of home automation) and I agree we are almost there
:-)

Cheers,

Eloy Paris.-

----- cut here; save as "ifttt2mh.php" -----
<?php
/*
 * IFTTT-to-MisterHouse Bridge.
 *
 * This PHP script receives a MisterHouse voice command (given to a
 * "Google Home" smart speaker, for example) from the IFTTT "Maker" channel,
 * performs some sanity checking, and then sends the command to MisterHouse
 * for execution via the mhsend utility.
 *
 * The script can also be invoked directly (i.e.
 * http://www.example.com/ifttt2mh.php), which helps for testing purposes.
 *
 * For this to work with the IFTTT "Maker" channel, configure your recipe's
 * action as follows:
 *
 * URL: https://www.example.com:4443/ifttt2mh.php
 *
 * Method: POST
 *
 * Content Type: application/x-www-form-urlencoded
 *
 * Body: auth=41af286dc0b172ed2f1ca934fd2278de4a1192302ffa07087cea2682e7d372e3&voice_cmd=<<<{{TextField}}>>>
 *
 *
 * Eloy Paris <peloy at chapus dot net>
 *
 * Fri Dec  2 13:41:15 EST 2016
 */

// Form handler
if ($_POST
    && isset($_POST['auth'], $_POST['voice_cmd']) ) {

    // Retrieve information from the HTTP POST
    $auth = $_POST['auth'];
    $voice_cmd = escapeshellcmd($_POST['voice_cmd']);

    $my_auth_token = '41af286dc0b172ed2f1ca934fd2278de4a1192302ffa07087cea2682e7d372e3';

    /* This will end up in the Apache error log; good for troubleshooting. */
    error_log("voice_cmd='$voice_cmd'");

    echo <<<EOT
<html>
<head>
<title>IFTTT-to-MisterHouse Bridge</title>
</head>

<body>
EOT;

    /*
     * This crappy "authentication" mechanism is better than nothing.
     */
    if ($auth != $my_auth_token) {
        echo <<<EOT
<p>Incorrect authentication token.</p>
</body>
</html>
EOT;
        exit;
    }

    $allowed_commands = array(
        '\w+ light (on|off)',
        'all lights off'
    );

    /*
     * Make sure the command that we received is allowed.
     */
    $allowed = false;

    foreach ($allowed_commands as $cmd_re) {
        if (preg_match("/$cmd_re/", $voice_cmd) ) {
            $allowed = true;
            break;
        }
    }

    if (!$allowed) {
        echo <<<EOT
<p>Command '$voice_cmd' is not allowed.</p>
</body>
</html>
EOT;
        exit;
    }

    /* Send the voice command to MisterHouse */
    exec("/opt/misterhouse/bin/mhsend -run '$voice_cmd'", $output, $ret);

    echo <<<EOT
<pre>
EOT;

    foreach ($output as $line) {
        echo "$line\n";
    }

    echo <<<EOT
</pre>

<p>ret = $ret</p>
</body>
</html>
EOT;

    /*
     * Here we could do some logging, e.g.
     *
     * mail("[hidden email]", $subject, $message, $headers);
     *
     * Or some redirection, e.g.
     *
     * header("Location: http://www.example.com:8080/newlocation.php");
     */

    exit;
}

    /*
     * We get here if the request is not a POST, i.e. if the user is just
     * using the form to test.
     */
?>

<html>
<head>
<title>IFTTT-to-MisterHouse Bridge</title>
</head>

<body>

<h1>IFTTT-to-MisterHouse Bridge</h1>

<p>This page is normally invoked from the Maker channel of the IFTTT
service. However, it can be invoked manually to test operation and
integration with MisterHouse. To use manually, enter your authentication
token and MisterHouse voice command below and then click on the 'Submit'
button.</p>

<form method="post" action="<?php echo htmlspecialchars($_SERVER['PHP_SELF']); ?>" id="submitform">
<p><input type="text" name="auth"></p>
<p><input type="text" name="voice_cmd"></p>
<p><input type="submit" value="Submit"></p>
</form>

</body>
</html>
----- cut here; save as "ifttt2mh.php" -----

> On Thu, Dec 1, 2016 at 8:53 AM, Eloy Paris <[hidden email]> wrote:
>
> > Hi Brian,
> >
> > On Fri, Nov 25, 2016 at 12:34:15PM -0600, Brian Klier wrote:
> >
> > > Eloy,
> > >
> > > I've been having some successes integrating Google Home/Assistant with
> > > MisterHouse, primarily due to some previous code Bruce Winter used to
> > > integrated Android Watches/Phones to MisterHouse through Tasker.
> > >
> > > Let me know if you have any questions or concerns.  I posted to the list
> > > earlier today with a fuzzy logic problem I've been encountering with this
> > > integration but exact matches have been working perfectly.  Very cool
> > stuff
> > > indeed!
> >
> > Thanks for the insight.
> >
> > I looked at your message to the mailing list (the one about the fuzzy
> > logic problem you are having) and at Bruce Winter's blog post on how
> > he integrated Android Watches/Phones to MisterHouse through Tasker,
> > and took some time to understand how you are integrating Google Home
> > with MisterHouse via IFTTT. Seems pretty straightforward, and I will be
> > testing your same approach as soon as I take the Google Home I purchased
> > out of the box.
> >
> > Regarding Google Home integration via IFTTT -- I am concerned about
> > exposing the MH web server to the Internet so the IFTTT Maker channel
> > can send voice commands to MH. Not only is it bad enough to expose
> > the MH web server to the Internet, but there isn't any type of
> > authentication in the /bin/runit.pl script that IFTTT talks to.
> > Hopefully the standard password_allow_clients mechanism applies and I'll
> > be able to at least specify a range of IPs from where the IFTTT request
> > is allowed to come from (if not I can accomplish something similar with
> > a firewall rule), but it still makes me very uncomfortable, plus the IP
> > range from where the IFTTT request will come from is not documented as
> > far as I can tell.
> >
> > In the end, I'd prefer a solution that does not involve an action (like
> > turning a light on or off) being initiated by a cloud service (like
> > IFTTT) but right now I don't see any other way. My hope is that if/when
> > Google decides to publish some type of API (perhaps "Direct Actions"
> > as mentioned at https://developers.google.com/actions/, once that is
> > available), we will be able to do something more "elegant" (though I am
> > not sure we'll be able to completely take the cloud out of the picture,
> > but we'll see).
> >
> > But I think that at least for now this Google Home integration via IFTTT
> > is good enough.
> >
> > Cheers,
> >
> > Eloy Paris.-
> >
> >

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
________________________________________________________
To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Google Home integration (was Re: Google Home on sale)

Greg Satz
Does google home, like the samsung TVs, listen in on everything being said and upload the recorded speech to a server "out there" for processing? So in effect you are bugging your own home? Or does this device locally process the speech recordings?

I haven't been very interested in enabling remote recording devices around my home so wondering what I may be missing or others aren't as concerned.

Thanks,
Greg

On Dec 2, 2016, at 11:44 PM, Eloy Paris wrote:

> Hi Brian,
>
> On Thu, Dec 01, 2016 at 09:58:31AM -0600, Brian Klier wrote:
>
>> Eloy,
>>
>> Great commentary!
>>
>> I can add that the standard password_allow_clients mechanism DOES apply for
>> the bin/runit.pl script, but unfortunately, I haven't found a specific IP
>> address range that incorporates all of IFTTT yet.  Clearly, IFTTT is not
>> using one specific IP address for these requests, and the IP address seems
>> to switch during each HTTP GET to MisterHouse.  One could of course rename
>> the bin/runit.pl script to something else, maybe with a long random number
>> string as part of the file name.  This would keep the nice guys nice, but
>> wouldn't help if the URL itself is being sniffed out.  I guess at this
>> point in time, if someone wants to turn on my Christmas lights, then I'll
>> deal with that when it happens!
>
> It looks like IFTTT uses Amazon Web Services so the HTTP request indeed
> comes from "random" addresses. It'd be hard to filter based on source IP
> address.
>
> I know there is a little bit of "security by obscurity" here because an
> "attacker" would have to know your public IP address, but someone who
> is determined could obtain that information (searching for one's name
> online, in social media, etc.).
>
> I do think the risk of someone wanting, and then able, to mess with our
> MisterHouse installations via this the IFTTT interface that we have
> been discussing is low. I think the risk is higher than just turning
> Christmas lights on, though -- any voice command can be executed, like
> "exit mister house", or "reboot the computer".
>
> In any case, I thought a little bit about how to improve the situation
> and came up with a slightly different solution that satisfies most of my
> requirements. I went about it a little differently...
>
> I already run an Apache web server that I expose to the Internet.
> In addition, I like PHP as a language to handle HTTP requests. So,
> instead of handling the incoming IFTTT request from the MisterHouse
> web server (and exposing the MisterHouse web server to the Internet),
> I wrote a small PHP script that is invoked from Apache that in the end
> executes the "mhsend" utility in MisterHouse's "bin" directory. Some
> enhancements/differences from the original runit.pl are:
>
> * A simple authentication mechanism -- an "authentication" token is sent
> in the request from IFTTT, which is then checked against a hardcoded
> token in the PHP script.
>
> * Using HTTP POST instead of HTTP GET -- that way the URL is kept short
> and we have more flexibility in terms of passing data from IFTTT.
>
> * A simple mechanism to filter out commands that are not allowed to run.
> For example, in my case I only allow at the moment "xxxxxx light
> (on|off)".
>
> * My Apache can serve over HTTPS, and the IFTTT Maker channel supports
> HTTPS, so the request comes in encrypted. Of course, it takes longer to
> establish an HTTPS session so I might just go with cleartext HTTP, but I
> wanted to test that HTTPS works.
>
> * No need to touch MisterHouse at all (no runit.pl, no MisterHouse web
> server); just use the existing MH voice commands.
>
> With these enhancements I feel comfortable opening my MisterHouse to
> voice commands received from IFTTT.
>
> (I'm including the script below, in case someone else already has a web
> server capable of running PHP scripts and wants to integrate IFTTT with
> MisterHouse. Instructions for the IFTTT recipe action are in the script
> itself. One future enhancement is to do fuzzy command matching, BTW.)
>
>> It's fun to have the stepping stones of Google Home<>MisterHouse
>> integration underway, security implications aside. Certainly, I'm
>> hoping there will be enough developers willing to help develop
>> built-in support for Google Home once Google releases the API this
>> month. I believe as more people integrate these voice assistant
>> technologies in their home, we are going to want to focus some
>> development effort on MisterHouse's "fuzzy logic", as it is far from
>> perfect at this point. Voice control truly is the crown jewel of home
>> automation control, and the future is nearly here!
>
> In terms of the Google Home -- I took it out of the box today and I
> have to say that I'm impressed: It became an immediate sensation in
> my household and it is not hard to see the immense potential that it
> has. It's so cool to be able to do so many things by just talking to it
> (adding items to the grocery shopping list, asking for music to play,
> asking for so many things like weather forecast, time, appointments,
> etc., and now also to send voice commands to MisterHouse).
>
> As you say, let's wait to see what can be done with the API once it is
> released but in the meantime I think that what we have via IFTTT is
> pretty decent. Pity the request comes from the "cloud" (which makes
> things a bit slow), but I can live with that.
>
> Voice control is definitely the crown jewel of home automation (I call
> it the 'Holy Grail' of home automation) and I agree we are almost there
> :-)
>
> Cheers,
>
> Eloy Paris.-
>
> ----- cut here; save as "ifttt2mh.php" -----
> <?php
> /*
> * IFTTT-to-MisterHouse Bridge.
> *
> * This PHP script receives a MisterHouse voice command (given to a
> * "Google Home" smart speaker, for example) from the IFTTT "Maker" channel,
> * performs some sanity checking, and then sends the command to MisterHouse
> * for execution via the mhsend utility.
> *
> * The script can also be invoked directly (i.e.
> * http://www.example.com/ifttt2mh.php), which helps for testing purposes.
> *
> * For this to work with the IFTTT "Maker" channel, configure your recipe's
> * action as follows:
> *
> * URL: https://www.example.com:4443/ifttt2mh.php
> *
> * Method: POST
> *
> * Content Type: application/x-www-form-urlencoded
> *
> * Body: auth=41af286dc0b172ed2f1ca934fd2278de4a1192302ffa07087cea2682e7d372e3&voice_cmd=<<<{{TextField}}>>>
> *
> *
> * Eloy Paris <peloy at chapus dot net>
> *
> * Fri Dec  2 13:41:15 EST 2016
> */
>
> // Form handler
> if ($_POST
>    && isset($_POST['auth'], $_POST['voice_cmd']) ) {
>
>    // Retrieve information from the HTTP POST
>    $auth = $_POST['auth'];
>    $voice_cmd = escapeshellcmd($_POST['voice_cmd']);
>
>    $my_auth_token = '41af286dc0b172ed2f1ca934fd2278de4a1192302ffa07087cea2682e7d372e3';
>
>    /* This will end up in the Apache error log; good for troubleshooting. */
>    error_log("voice_cmd='$voice_cmd'");
>
>    echo <<<EOT
> <html>
> <head>
> <title>IFTTT-to-MisterHouse Bridge</title>
> </head>
>
> <body>
> EOT;
>
>    /*
>     * This crappy "authentication" mechanism is better than nothing.
>     */
>    if ($auth != $my_auth_token) {
> echo <<<EOT
> <p>Incorrect authentication token.</p>
> </body>
> </html>
> EOT;
> exit;
>    }
>
>    $allowed_commands = array(
> '\w+ light (on|off)',
> 'all lights off'
>    );
>
>    /*
>     * Make sure the command that we received is allowed.
>     */
>    $allowed = false;
>
>    foreach ($allowed_commands as $cmd_re) {
> if (preg_match("/$cmd_re/", $voice_cmd) ) {
>    $allowed = true;
>    break;
> }
>    }
>
>    if (!$allowed) {
> echo <<<EOT
> <p>Command '$voice_cmd' is not allowed.</p>
> </body>
> </html>
> EOT;
> exit;
>    }
>
>    /* Send the voice command to MisterHouse */
>    exec("/opt/misterhouse/bin/mhsend -run '$voice_cmd'", $output, $ret);
>
>    echo <<<EOT
> <pre>
> EOT;
>
>    foreach ($output as $line) {
> echo "$line\n";
>    }
>
>    echo <<<EOT
> </pre>
>
> <p>ret = $ret</p>
> </body>
> </html>
> EOT;
>
>    /*
>     * Here we could do some logging, e.g.
>     *
>     * mail("[hidden email]", $subject, $message, $headers);
>     *
>     * Or some redirection, e.g.
>     *
>     * header("Location: http://www.example.com:8080/newlocation.php");
>     */
>
>    exit;
> }
>
>    /*
>     * We get here if the request is not a POST, i.e. if the user is just
>     * using the form to test.
>     */
> ?>
>
> <html>
> <head>
> <title>IFTTT-to-MisterHouse Bridge</title>
> </head>
>
> <body>
>
> <h1>IFTTT-to-MisterHouse Bridge</h1>
>
> <p>This page is normally invoked from the Maker channel of the IFTTT
> service. However, it can be invoked manually to test operation and
> integration with MisterHouse. To use manually, enter your authentication
> token and MisterHouse voice command below and then click on the 'Submit'
> button.</p>
>
> <form method="post" action="<?php echo htmlspecialchars($_SERVER['PHP_SELF']); ?>" id="submitform">
> <p><input type="text" name="auth"></p>
> <p><input type="text" name="voice_cmd"></p>
> <p><input type="submit" value="Submit"></p>
> </form>
>
> </body>
> </html>
> ----- cut here; save as "ifttt2mh.php" -----
>
>> On Thu, Dec 1, 2016 at 8:53 AM, Eloy Paris <[hidden email]> wrote:
>>
>>> Hi Brian,
>>>
>>> On Fri, Nov 25, 2016 at 12:34:15PM -0600, Brian Klier wrote:
>>>
>>>> Eloy,
>>>>
>>>> I've been having some successes integrating Google Home/Assistant with
>>>> MisterHouse, primarily due to some previous code Bruce Winter used to
>>>> integrated Android Watches/Phones to MisterHouse through Tasker.
>>>>
>>>> Let me know if you have any questions or concerns.  I posted to the list
>>>> earlier today with a fuzzy logic problem I've been encountering with this
>>>> integration but exact matches have been working perfectly.  Very cool
>>> stuff
>>>> indeed!
>>>
>>> Thanks for the insight.
>>>
>>> I looked at your message to the mailing list (the one about the fuzzy
>>> logic problem you are having) and at Bruce Winter's blog post on how
>>> he integrated Android Watches/Phones to MisterHouse through Tasker,
>>> and took some time to understand how you are integrating Google Home
>>> with MisterHouse via IFTTT. Seems pretty straightforward, and I will be
>>> testing your same approach as soon as I take the Google Home I purchased
>>> out of the box.
>>>
>>> Regarding Google Home integration via IFTTT -- I am concerned about
>>> exposing the MH web server to the Internet so the IFTTT Maker channel
>>> can send voice commands to MH. Not only is it bad enough to expose
>>> the MH web server to the Internet, but there isn't any type of
>>> authentication in the /bin/runit.pl script that IFTTT talks to.
>>> Hopefully the standard password_allow_clients mechanism applies and I'll
>>> be able to at least specify a range of IPs from where the IFTTT request
>>> is allowed to come from (if not I can accomplish something similar with
>>> a firewall rule), but it still makes me very uncomfortable, plus the IP
>>> range from where the IFTTT request will come from is not documented as
>>> far as I can tell.
>>>
>>> In the end, I'd prefer a solution that does not involve an action (like
>>> turning a light on or off) being initiated by a cloud service (like
>>> IFTTT) but right now I don't see any other way. My hope is that if/when
>>> Google decides to publish some type of API (perhaps "Direct Actions"
>>> as mentioned at https://developers.google.com/actions/, once that is
>>> available), we will be able to do something more "elegant" (though I am
>>> not sure we'll be able to completely take the cloud out of the picture,
>>> but we'll see).
>>>
>>> But I think that at least for now this Google Home integration via IFTTT
>>> is good enough.
>>>
>>> Cheers,
>>>
>>> Eloy Paris.-
>>>
>>>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> ________________________________________________________
> To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users
>


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
________________________________________________________
To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Google Home integration (was Re: Google Home on sale)

Brian Klier
You can put a packet sniffer on your network or look at router logs to verify this, but anything that is said after "OK Google" or "Hey Google" (or an errant trigger of this keyphrase) is uploaded to Google's servers for processing.  The mic is always "on", but does not send anything to Google's servers until it hears the wakeup phrase.

You can actually listen to what commands Google is picking up in your house by going to https://myactivity.google.com/myactivity (This will work if you have an Android phone and use OK Google as well).

By the way Eloy, thanks again for the excellent script!  It will need some adapting to work on Windows, but if anyone is particularly concerned about the security of their MisterHouse installation when integrating with Google Home, your script is certainly the place to start!

Brian

On Sat, Dec 3, 2016 at 9:47 AM, Greg Satz <[hidden email]> wrote:
Does google home, like the samsung TVs, listen in on everything being said and upload the recorded speech to a server "out there" for processing? So in effect you are bugging your own home? Or does this device locally process the speech recordings?

I haven't been very interested in enabling remote recording devices around my home so wondering what I may be missing or others aren't as concerned.

Thanks,
Greg

On Dec 2, 2016, at 11:44 PM, Eloy Paris wrote:

> Hi Brian,
>
> On Thu, Dec 01, 2016 at 09:58:31AM -0600, Brian Klier wrote:
>
>> Eloy,
>>
>> Great commentary!
>>
>> I can add that the standard password_allow_clients mechanism DOES apply for
>> the bin/runit.pl script, but unfortunately, I haven't found a specific IP
>> address range that incorporates all of IFTTT yet.  Clearly, IFTTT is not
>> using one specific IP address for these requests, and the IP address seems
>> to switch during each HTTP GET to MisterHouse.  One could of course rename
>> the bin/runit.pl script to something else, maybe with a long random number
>> string as part of the file name.  This would keep the nice guys nice, but
>> wouldn't help if the URL itself is being sniffed out.  I guess at this
>> point in time, if someone wants to turn on my Christmas lights, then I'll
>> deal with that when it happens!
>
> It looks like IFTTT uses Amazon Web Services so the HTTP request indeed
> comes from "random" addresses. It'd be hard to filter based on source IP
> address.
>
> I know there is a little bit of "security by obscurity" here because an
> "attacker" would have to know your public IP address, but someone who
> is determined could obtain that information (searching for one's name
> online, in social media, etc.).
>
> I do think the risk of someone wanting, and then able, to mess with our
> MisterHouse installations via this the IFTTT interface that we have
> been discussing is low. I think the risk is higher than just turning
> Christmas lights on, though -- any voice command can be executed, like
> "exit mister house", or "reboot the computer".
>
> In any case, I thought a little bit about how to improve the situation
> and came up with a slightly different solution that satisfies most of my
> requirements. I went about it a little differently...
>
> I already run an Apache web server that I expose to the Internet.
> In addition, I like PHP as a language to handle HTTP requests. So,
> instead of handling the incoming IFTTT request from the MisterHouse
> web server (and exposing the MisterHouse web server to the Internet),
> I wrote a small PHP script that is invoked from Apache that in the end
> executes the "mhsend" utility in MisterHouse's "bin" directory. Some
> enhancements/differences from the original runit.pl are:
>
> * A simple authentication mechanism -- an "authentication" token is sent
> in the request from IFTTT, which is then checked against a hardcoded
> token in the PHP script.
>
> * Using HTTP POST instead of HTTP GET -- that way the URL is kept short
> and we have more flexibility in terms of passing data from IFTTT.
>
> * A simple mechanism to filter out commands that are not allowed to run.
> For example, in my case I only allow at the moment "xxxxxx light
> (on|off)".
>
> * My Apache can serve over HTTPS, and the IFTTT Maker channel supports
> HTTPS, so the request comes in encrypted. Of course, it takes longer to
> establish an HTTPS session so I might just go with cleartext HTTP, but I
> wanted to test that HTTPS works.
>
> * No need to touch MisterHouse at all (no runit.pl, no MisterHouse web
> server); just use the existing MH voice commands.
>
> With these enhancements I feel comfortable opening my MisterHouse to
> voice commands received from IFTTT.
>
> (I'm including the script below, in case someone else already has a web
> server capable of running PHP scripts and wants to integrate IFTTT with
> MisterHouse. Instructions for the IFTTT recipe action are in the script
> itself. One future enhancement is to do fuzzy command matching, BTW.)
>
>> It's fun to have the stepping stones of Google Home<>MisterHouse
>> integration underway, security implications aside. Certainly, I'm
>> hoping there will be enough developers willing to help develop
>> built-in support for Google Home once Google releases the API this
>> month. I believe as more people integrate these voice assistant
>> technologies in their home, we are going to want to focus some
>> development effort on MisterHouse's "fuzzy logic", as it is far from
>> perfect at this point. Voice control truly is the crown jewel of home
>> automation control, and the future is nearly here!
>
> In terms of the Google Home -- I took it out of the box today and I
> have to say that I'm impressed: It became an immediate sensation in
> my household and it is not hard to see the immense potential that it
> has. It's so cool to be able to do so many things by just talking to it
> (adding items to the grocery shopping list, asking for music to play,
> asking for so many things like weather forecast, time, appointments,
> etc., and now also to send voice commands to MisterHouse).
>
> As you say, let's wait to see what can be done with the API once it is
> released but in the meantime I think that what we have via IFTTT is
> pretty decent. Pity the request comes from the "cloud" (which makes
> things a bit slow), but I can live with that.
>
> Voice control is definitely the crown jewel of home automation (I call
> it the 'Holy Grail' of home automation) and I agree we are almost there
> :-)
>
> Cheers,
>
> Eloy Paris.-
>
> ----- cut here; save as "ifttt2mh.php" -----
> <?php
> /*
> * IFTTT-to-MisterHouse Bridge.
> *
> * This PHP script receives a MisterHouse voice command (given to a
> * "Google Home" smart speaker, for example) from the IFTTT "Maker" channel,
> * performs some sanity checking, and then sends the command to MisterHouse
> * for execution via the mhsend utility.
> *
> * The script can also be invoked directly (i.e.
> * http://www.example.com/ifttt2mh.php), which helps for testing purposes.
> *
> * For this to work with the IFTTT "Maker" channel, configure your recipe's
> * action as follows:
> *
> * URL: https://www.example.com:4443/ifttt2mh.php
> *
> * Method: POST
> *
> * Content Type: application/x-www-form-urlencoded
> *
> * Body: auth=41af286dc0b172ed2f1ca934fd2278de4a1192302ffa07087cea2682e7d372e3&voice_cmd=<<<{{TextField}}>>>
> *
> *
> * Eloy Paris <peloy at chapus dot net>
> *
> * Fri Dec  2 13:41:15 EST 2016
> */
>
> // Form handler
> if ($_POST
>    && isset($_POST['auth'], $_POST['voice_cmd']) ) {
>
>    // Retrieve information from the HTTP POST
>    $auth = $_POST['auth'];
>    $voice_cmd = escapeshellcmd($_POST['voice_cmd']);
>
>    $my_auth_token = '41af286dc0b172ed2f1ca934fd2278de4a1192302ffa07087cea2682e7d372e3';
>
>    /* This will end up in the Apache error log; good for troubleshooting. */
>    error_log("voice_cmd='$voice_cmd'");
>
>    echo <<<EOT
> <html>
> <head>
> <title>IFTTT-to-MisterHouse Bridge</title>
> </head>
>
> <body>
> EOT;
>
>    /*
>     * This crappy "authentication" mechanism is better than nothing.
>     */
>    if ($auth != $my_auth_token) {
>       echo <<<EOT
> <p>Incorrect authentication token.</p>
> </body>
> </html>
> EOT;
>       exit;
>    }
>
>    $allowed_commands = array(
>       '\w+ light (on|off)',
>       'all lights off'
>    );
>
>    /*
>     * Make sure the command that we received is allowed.
>     */
>    $allowed = false;
>
>    foreach ($allowed_commands as $cmd_re) {
>       if (preg_match("/$cmd_re/", $voice_cmd) ) {
>           $allowed = true;
>           break;
>       }
>    }
>
>    if (!$allowed) {
>       echo <<<EOT
> <p>Command '$voice_cmd' is not allowed.</p>
> </body>
> </html>
> EOT;
>       exit;
>    }
>
>    /* Send the voice command to MisterHouse */
>    exec("/opt/misterhouse/bin/mhsend -run '$voice_cmd'", $output, $ret);
>
>    echo <<<EOT
> <pre>
> EOT;
>
>    foreach ($output as $line) {
>       echo "$line\n";
>    }
>
>    echo <<<EOT
> </pre>
>
> <p>ret = $ret</p>
> </body>
> </html>
> EOT;
>
>    /*
>     * Here we could do some logging, e.g.
>     *
>     * mail("[hidden email]", $subject, $message, $headers);
>     *
>     * Or some redirection, e.g.
>     *
>     * header("Location: http://www.example.com:8080/newlocation.php");
>     */
>
>    exit;
> }
>
>    /*
>     * We get here if the request is not a POST, i.e. if the user is just
>     * using the form to test.
>     */
> ?>
>
> <html>
> <head>
> <title>IFTTT-to-MisterHouse Bridge</title>
> </head>
>
> <body>
>
> <h1>IFTTT-to-MisterHouse Bridge</h1>
>
> <p>This page is normally invoked from the Maker channel of the IFTTT
> service. However, it can be invoked manually to test operation and
> integration with MisterHouse. To use manually, enter your authentication
> token and MisterHouse voice command below and then click on the 'Submit'
> button.</p>
>
> <form method="post" action="<?php echo htmlspecialchars($_SERVER['PHP_SELF']); ?>" id="submitform">
> <p><input type="text" name="auth"></p>
> <p><input type="text" name="voice_cmd"></p>
> <p><input type="submit" value="Submit"></p>
> </form>
>
> </body>
> </html>
> ----- cut here; save as "ifttt2mh.php" -----
>
>> On Thu, Dec 1, 2016 at 8:53 AM, Eloy Paris <[hidden email]> wrote:
>>
>>> Hi Brian,
>>>
>>> On Fri, Nov 25, 2016 at 12:34:15PM -0600, Brian Klier wrote:
>>>
>>>> Eloy,
>>>>
>>>> I've been having some successes integrating Google Home/Assistant with
>>>> MisterHouse, primarily due to some previous code Bruce Winter used to
>>>> integrated Android Watches/Phones to MisterHouse through Tasker.
>>>>
>>>> Let me know if you have any questions or concerns.  I posted to the list
>>>> earlier today with a fuzzy logic problem I've been encountering with this
>>>> integration but exact matches have been working perfectly.  Very cool
>>> stuff
>>>> indeed!
>>>
>>> Thanks for the insight.
>>>
>>> I looked at your message to the mailing list (the one about the fuzzy
>>> logic problem you are having) and at Bruce Winter's blog post on how
>>> he integrated Android Watches/Phones to MisterHouse through Tasker,
>>> and took some time to understand how you are integrating Google Home
>>> with MisterHouse via IFTTT. Seems pretty straightforward, and I will be
>>> testing your same approach as soon as I take the Google Home I purchased
>>> out of the box.
>>>
>>> Regarding Google Home integration via IFTTT -- I am concerned about
>>> exposing the MH web server to the Internet so the IFTTT Maker channel
>>> can send voice commands to MH. Not only is it bad enough to expose
>>> the MH web server to the Internet, but there isn't any type of
>>> authentication in the /bin/runit.pl script that IFTTT talks to.
>>> Hopefully the standard password_allow_clients mechanism applies and I'll
>>> be able to at least specify a range of IPs from where the IFTTT request
>>> is allowed to come from (if not I can accomplish something similar with
>>> a firewall rule), but it still makes me very uncomfortable, plus the IP
>>> range from where the IFTTT request will come from is not documented as
>>> far as I can tell.
>>>
>>> In the end, I'd prefer a solution that does not involve an action (like
>>> turning a light on or off) being initiated by a cloud service (like
>>> IFTTT) but right now I don't see any other way. My hope is that if/when
>>> Google decides to publish some type of API (perhaps "Direct Actions"
>>> as mentioned at https://developers.google.com/actions/, once that is
>>> available), we will be able to do something more "elegant" (though I am
>>> not sure we'll be able to completely take the cloud out of the picture,
>>> but we'll see).
>>>
>>> But I think that at least for now this Google Home integration via IFTTT
>>> is good enough.
>>>
>>> Cheers,
>>>
>>> Eloy Paris.-
>>>
>>>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> ________________________________________________________
> To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users
>



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
________________________________________________________
To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Google Home integration (was Re: Google Home on sale)

Neil Cherry-3
In reply to this post by Greg Satz
This is one of the reasons I intend to build the Rapsberry Pi version
of these devices. It gives a bit more control over when it's being
sent. The rest is still being processed but the service (Google,
Amazon, etc.).

--
Linux Home Automation         Neil Cherry       [hidden email]
http://www.linuxha.com/                         Main site
http://linuxha.blogspot.com/                    My HA Blog
Author of:     Linux Smart Homes For Dummies

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
________________________________________________________
To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Google Home integration (was Re: Google Home on sale)

Eloy Paris
In reply to this post by Brian Klier
On Sat, Dec 03, 2016 at 09:58:22AM -0600, Brian Klier wrote:

> You can put a packet sniffer on your network or look at router logs
> to verify this, but anything that is said after "OK Google" or "Hey
> Google" (or an errant trigger of this keyphrase) is uploaded to
> Google's servers for processing. The mic is always "on", but does not
> send anything to Google's servers until it hears the wakeup phrase.

Yes, everything seems to be sent to the Google cloud, even something as
basic as "hey Google, what time is it?".

> You can actually listen to what commands Google is picking up in your
> house by going to https://myactivity.google.com/myactivity (This will
> work if you have an Android phone and use OK Google as well).

Thanks for the pointer; I had not seen the specific information they
keep about the stored voice records. From a privacy point of view I am
not too worried about the information (voice interactions) that gets
sent to the cloud and stored on the Google cloud; I'm more concerned
about the device listening in without me being aware but that does not
seem to be the case. Obviously, it'd be better if nothing gets sent to
the cloud and everything happens locally, but I think that defeats the
purpose of this product, which is to make the user's life easier by
interacting with cloud services from Google and other companies.

> By the way Eloy, thanks again for the excellent script! It will need
> some adapting to work on Windows, but if anyone is particularly
> concerned about the security of their MisterHouse installation when
> integrating with Google Home, your script is certainly the place to
> start!

You're welcome!

I haven't looked at what it'd take to make the PHP script run on Windows
but PHP works on Windows, apparently even with IIS, and MisterHouse
ships with a mhsend.bat script for Windows (in the bin/ directory) that
the PHP script can call (I'm planning to remove the need for mhsend by
connecting directly from PHP to the MH control port as an optimization,
so there won't be a need for mhsend in the future anyway). And if
nothing works, a Raspberry Pi to run a web server with PHP is cheap ;-)

On a related note, I added fuzzy matching to my script. I am using
Levenshtein distance and also running into the issue you brought up,
i.e. "turn off the kitchen light" gets matched to "kitchen light on".
Saying the exact match "kitchen light off" is no problem but the
variation is giving that problem, which is obviously not user friendly.
I am still not sure how to go about it. I found online someone's
algorithm that was handling well that case but then it was misfiring
with "turn play room light off" (matches to "garage hallway light off")
so I am back to plain Levenshtein. I am tempted to manually add more
natural language versions of the auto-generated voice commands, i.e.
manually add "turn {off,on} the kitchen light" and "turn the kitchen
light {off,on}" to get rid of the incorrect fuzzy matches.

I also changed the way to trigger my IFTTT applet -- I was originally
using "okay google, turn XXXX" but that proved to be too limiting. For
example, what about "okay google, close the garage door"? -- I would
have needed another IFTTT applet for that, and then what about opening
the garage door; another applet for that? Doesn't scale. It also was in
conflict with some other things not related to MisterHouse (for example,
it conflicts with "okay google, turn the volume up"). Now my IFTTT
trigger is "okay google, tell house to XXXX" (I think like in Bruce's
demo a few months ago). That seems to be working well so far.

All in all, I think things are working very well in terms of voice
commands and interaction between Google Voice and MisterHouse via IFTTT;
thank you for the pointers! (BTW, my lights are controlled by INSTEON.)

Cheers,

Eloy Paris.-


> On Sat, Dec 3, 2016 at 9:47 AM, Greg Satz <[hidden email]> wrote:
>
> > Does google home, like the samsung TVs, listen in on everything being said
> > and upload the recorded speech to a server "out there" for processing? So
> > in effect you are bugging your own home? Or does this device locally
> > process the speech recordings?
> >
> > I haven't been very interested in enabling remote recording devices around
> > my home so wondering what I may be missing or others aren't as concerned.
> >
> > Thanks,
> > Greg
> >

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
________________________________________________________
To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Google Home integration (was Re: Google Home on sale)

H Plato
In reply to this post by Neil Cherry-3
I backed the mycroft project on kickstarter thats building a raspberry PI based open-source natural language processor. It still uses ‘cloud APIs’, but this might be a more adaptable solution to the workarounds for Siri, Google and Amazon.

https://www.kickstarter.com/projects/aiforeveryone/mycroft-an-open-source-artificial-intelligence-for

They haven’t shipped yet, when/if I get one I’ll see how it can be integrated.

> On Dec 3, 2016, at 10:14 AM, Neil Cherry <[hidden email]> wrote:
>
> This is one of the reasons I intend to build the Rapsberry Pi version
> of these devices. It gives a bit more control over when it's being
> sent. The rest is still being processed but the service (Google,
> Amazon, etc.).
>
> --
> Linux Home Automation         Neil Cherry       [hidden email]
> http://www.linuxha.com/                         Main site
> http://linuxha.blogspot.com/                    My HA Blog
> Author of:     Linux Smart Homes For Dummies
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> ________________________________________________________
> To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users
>


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
________________________________________________________
To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Google Home integration (was Re: Google Home on sale)

Brian Klier
In reply to this post by Eloy Paris
It almost seems that we need something like the Levenshtein distance method, but with additional higher values being added to certain words like "yes" and "no", to ensure that those match under every circumstance.  Also, perhaps we could ignore common verbs before it runs through the algorithms?  All we really need is the item name, and its state.  Cut out the rest of the words on the interpreted speech, and cut out those words in the MH voice commands when looking for the match.  I think we'll get better accuracy this way (but I haven't tested this yet). I actually did add a lot more voice command variants to try to solve my initial problem, but it seems the Levenshtein method gets more and more confused with the more voice commands it has to run through, and it particularly favors short 2 or 3 word phrases for every match regardless of what the Google Home interpreted command is.

Fortunately, the Google Home API was released today :-)

Check it out:


Brian

On Sun, Dec 4, 2016 at 11:55 AM, Eloy Paris <[hidden email]> wrote:


On a related note, I added fuzzy matching to my script. I am using
Levenshtein distance and also running into the issue you brought up,
i.e. "turn off the kitchen light" gets matched to "kitchen light on".
Saying the exact match "kitchen light off" is no problem but the
variation is giving that problem, which is obviously not user friendly.
I am still not sure how to go about it. I found online someone's
algorithm that was handling well that case but then it was misfiring
with "turn play room light off" (matches to "garage hallway light off")
so I am back to plain Levenshtein. I am tempted to manually add more
natural language versions of the auto-generated voice commands, i.e.
manually add "turn {off,on} the kitchen light" and "turn the kitchen
light {off,on}" to get rid of the incorrect fuzzy matches.

Cheers,

Eloy Paris.-



------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/xeonphi
________________________________________________________
To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Google Home integration (was Re: Google Home on sale)

Eloy Paris
Hi Brian,

On Thu, Dec 08, 2016 at 05:09:13PM -0600, Brian Klier wrote:

[...]

> Fortunately, the Google Home API was released today :-)
>
> Check it out:
>
> https://developers.google.com/actions/

Yes, I saw that earlier today. It's pretty cool. It seems to me like
they really have put a lot of thought into it. I'll start reading,
watching the videos, and playing with the examples to see if there's
something that can be done without too much effort (I'm a hack as a
programmer so if this requires learning a lot of new stuff then I think
I'll have to wait for someone else to come up with something cool).

Unfortunately they haven't opened direct actions yet; just
conversational actions. But that could be good because conversational
actions will allow for more (at the expense of more work from the
developer, though).

Cheers,

Eloy Paris.-


------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/xeonphi
________________________________________________________
To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users

Loading...