Quantcast
Channel: Raspberry Pi Forums
Viewing all articles
Browse latest Browse all 8037

General • Re: TFLite micro on RP2040

$
0
0
I think I narrowed down what the problem is (or one of them). It appears unrelated to tflite micro, perhaps I should make it its own post:

Arduino has this

Code:

void CaptureSamples() {  // This is how many bytes of new data we have each time this is called  const int number_of_samples = DEFAULT_PDM_BUFFER_SIZE/2;  // Calculate what timestamp the last audio sample represents  const int32_t time_in_ms =      g_latest_audio_timestamp +      (number_of_samples / (kAudioSampleFrequency / 1000));  // Determine the index, in the history of all samples, of the last sample  const int32_t start_sample_offset =      g_latest_audio_timestamp * (kAudioSampleFrequency / 1000);  // Determine the index of this sample in our ring buffer  const int capture_index = start_sample_offset % kAudioCaptureBufferSize;  // Read the data to the correct place in our buffer  PDM.read(g_audio_capture_buffer + capture_index, DEFAULT_PDM_BUFFER_SIZE);  // This is how we let the outside world know that new audio data has arrived.  g_latest_audio_timestamp = time_in_ms;}TfLiteStatus InitAudioRecording(tflite::ErrorReporter* error_reporter) {  e = error_reporter;  // Hook up the callback that will be called with each sample  PDM.onReceive(CaptureSamples);  // Start listening for audio: MONO @ 16KHz with gain at 20  PDM.begin(1, kAudioSampleFrequency);  PDM.setGain(20);  // Block until we have our first audio sample  while (!g_latest_audio_timestamp) {  }  return kTfLiteOk;}
So when samples are ready from PDM, grab the contents of the buffer and update our timestamp by num_samples/sampling rate, all in ms. An example from recording is from 5024ms to 5168ms - 144ms. Buffer size is 512 so we're getting 256 samples, in 144ms - that's one sample every ~0.5ms. That's a sampling rate of 2kHz? Whereas kAudioSampleFrequency is 16kHz in the code. I'm confused here. I can only imagine behind the scenes that some filters are applied, etc etc

Compare this to the RP2040 (i tried 2350, same results):
all constants are the same, so buffer size is 512, num_samples is buiffer/2 == 256, kAudioSamplingF = 16000
Code is the same

Code:

void CaptureSamples() {  // This is how many bytes of new data we have each time this is called  const int number_of_samples = DEFAULT_PDM_BUFFER_SIZE/2;  // Calculate what timestamp the last audio sample represents  const int32_t time_in_ms =      g_latest_audio_timestamp +      (number_of_samples / (kAudioSampleFrequency / 1000));  // Determine the index, in the history of all samples, of the last sample  const int32_t start_sample_offset =      g_latest_audio_timestamp * (kAudioSampleFrequency / 1000);  // Determine the index of this sample in our ring buffer  const int capture_index = start_sample_offset % kAudioCaptureBufferSize;  pdm_microphone_read(g_audio_capture_buffer + capture_index, DEFAULT_PDM_BUFFER_SIZE);  printf("Audio update at time %d from time %d with %d samples\n", time_in_ms, g_latest_audio_timestamp, number_of_samples);  // This is how we let the outside world know that new audio data has arrived.  g_latest_audio_timestamp = time_in_ms;}
This prints
Audio update at time 1248 from time 1232 with 256 samples
so 16ms, with 256 samples. This equates to 0.0625ms a sample, or ... well, precisely 16kHz.

Perhaps the RP2040 is skipping the filters - after all, these are baked onto the nRF chip of the Arduino, and are in software on the RP.
Nope: https://github.com/ArmDeveloperEcosyste ... one.c#L220
This filter appears to be happening live.
From what I've read this is the same filtering that happens on the nRF, so I don't understand why I'm getting such different results.

Statistics: Posted by mckinnonbuilding — Mon May 19, 2025 4:00 am



Viewing all articles
Browse latest Browse all 8037

Trending Articles