GRPC Pushtotalk hangs after a few secs of inactivity

[Whuzz]

System: Raspberry Pi 3 + Matrix Creator (for mics)
OS: Debian Stretch
Using portaudio, no pulseaudio

When I launch the pushtotalk.py, everything works fine as long as I keep activating (pressing enter) and talking. I can perform several requests and will get proper responses. Mics work, speech works, all good.
But whenever I leave the program waiting for input ("Press Enter to send a new request...") for longer than a few seconds it will hang on the next activation.
This will happen every time, even when no request was made before. In the code below, I used ctrl+C once it froze. Then another ctrl+C after receiving the traceback.

After that, the client doesn't seem to successfully exit. Sometimes, the system completely freezes up (hard reboot required), but even it doesn't, the Pi refuses to properly reboot. If I try, the system will hang during shutdown, again requiring a hard reboot.

Log:

DEBUG:google.auth.transport.requests:Making request: POST https://accounts.google.com/o/oauth2/token
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): accounts.google.com
DEBUG:urllib3.connectionpool:https://accounts.google.com:443 "POST /o/oauth2/token HTTP/1.1" 200 None
INFO:root:Connecting to embeddedassistant.googleapis.com
INFO:root:Using device model xxx-matrix-creator-xxx and device id xxxx-3a88-11e8-9d84-b82xxxxx
Press Enter to send a new request...
INFO:root:Recording audio request.
DEBUG:root:AssistRequest: config {
  audio_in_config {
    encoding: LINEAR16
    sample_rate_hertz: 16000
  }
  audio_out_config {
    encoding: LINEAR16
    sample_rate_hertz: 16000
    volume_percentage: 50
  }
  dialog_state_in {
    language_code: "en-US"
  }
  device_config {
    device_id: "xxxxx-3a88-11e8-9d84-b827xxxxx"
    device_model_id: "xxxx-matrix-creator-main-xxxx"
  }
}
^C
Aborted!
^CException ignored in: <module 'threading' from '/usr/lib/python3.5/threading.py'>
Traceback (most recent call last):
  File "/usr/lib/python3.5/threading.py", line 1288, in _shutdown
    t.join()
  File "/usr/local/lib/python3.5/dist-packages/grpc/_common.py", line 135, in join
    super(CleanupThread, self).join(timeout)
  File "/usr/lib/python3.5/threading.py", line 1054, in join
    self._wait_for_tstate_lock()
  File "/usr/lib/python3.5/threading.py", line 1070, in _wait_for_tstate_lock
    elif lock.acquire(block, timeout):
KeyboardInterrupt
^C

My Alsa .asoundrc:

pcm.!default {
  type asym
  capture.pcm "mic"
  playback.pcm "speaker"
}

pcm.mic {
   type softvol
   slave {
     pcm "array"
   }
   control {
     name "MicArray Master"
        card 0
   }
}

pcm.speaker {
  type plug
  slave {
    pcm "hw:0,0"
  }
}

pcm.array {
  type plug
  slave {
    pcm "hw:2,0"
  }
}

[Whuzz]

This is getting more and more strange...
I rewrote audio_helpers to work with pyalsaaudio instead of sounddevice, since I thought the problem was related to sounddevice / portaudio. However, I get the exact same behavior. :(

[moham96]

So the problem is in the
click.pause(info='Press Enter to send a new request...')
try using a different trigger method (GPIO) or try
input('Press Enter to send a new request...')
Also, try to ditch click library altogether (supply the argument to the main function inside the script)

[Whuzz]

@moham96 I'm not sure the problem is there, but maybe the "halting" donw by click has something to with it. I will try some alternate trigger methods like you suggest.

Edit: Just tried with input instead of click. Exact same behavior.

I have narrowed it down to line 195 in audio_helpers.py:
buf, overflow = self._audio_stream.read(size)

So as soon as the stream from the mic is being read (doesn't matter if it is done through sounddevice or alsaaudio) after >10 secs (or so) of being halted by click, I get a crash.

[moham96]

@Whuzz So maybe It is reading and filling the ram? try to watch the process memory consumption(maybe use htop and sort the result by mem), see if it's reaching 100% of mem and then cause the crash.

Also, try to set the memory split to 16 in the raspi-config

[Whuzz]

@moham96 CPU use is below 5%, RAM use never above 35MB (of the python process), even at the point it hangs. Total RAM used is only 120MB out of 1 GB. So that doesn't seem to be the issue.

Since no one else seems to be suffering from this, I suspect it must have something to with the Matrix Creator's HAL / Kernel modules. Not sure where to look next though :(

[moham96]

@Whuzz well that's easy to check, Try to use another mic(cheap USB mic or an old webcam will do) and see if the problem get's solved

[Whuzz]

@moham96 I'll do that later yeah, just as a final check. For now I've solved it with a (super dirty) workaround. While waiting on a keypress, I open a seperate thread that keeps reading from the mic. That seems to prevent something from overrunning, or closing, while the system is waiting.

In the end, I hope to solve this in a more permanent way, because I intend to use snowboy (Kitt.ai) for custom wakeword activation. For that, I need the mic constant open anyway.