TR-595E Custom Tricorder

[JaredTS486]

Anyone programming on the Pico, this was an essential link to the process.
Using this core with PlatformIO — Arduino-Pico 3.1.1 documentation

I tried using this Adafruit I2S to DAC decoder for audio with disappointing results. The Pico seems to struggle processing audio, (WAV, AAC, etc) from PROGMEM in the same way that the TFT sprites do. Even disabling the TFT code altogether and increasing the RP2040 frequency still outputs choppy audio. I am using this ESP8266Audio library at the moment.

Using one of those Adafruit FX sound boards was probably the right way to go. It sounds like there are some ESP32/ESP8266 boards that should be able to do both the TFT display and audio but I have not tried any yet. If anyone has any suggestions on proper I2S output on the Pico let me know!

 

[wystewart]

Anyone programming on the Pico, this was an essential link to the process.
Using this core with PlatformIO — Arduino-Pico 3.1.1 documentation

I tried using this Adafruit I2S to DAC decoder for audio with disappointing results. The Pico seems to struggle processing audio, (WAV, AAC, etc) from PROGMEM in the same way that the TFT sprites do. Even disabling the TFT code altogether and increasing the RP2040 frequency still outputs choppy audio. I am using this ESP8266Audio library at the moment.

Using one of those Adafruit FX sound boards was probably the right way to go. It sounds like there are some ESP32/ESP8266 boards that should be able to do both the TFT display and audio but I have not tried any yet. If anyone has any suggestions on proper I2S output on the Pico let me know!

yes I did try at first to use a board with RP2040 processor. I also gave up. The I2S audio works great in circuitpython on RP2040, but I could not get it to work well in Ardunio. I tried several different audio libraries including modifying them. The problem is that the RP2040 does not have a real I2S port, it is emulated. Use the ESP32S3 instead it has a real I2S port, it works way better, more options and more memory available. On top of that dual core. I was using that it was working fine, until I decided I needed more, now with a STM32H750 processor because I needed more IO and more horsepower for the LCD. But I would really recommend the ESP32S3 really easy to use and get running, very well supported in Arduino.. The STM32 on the other hand I'm still working on getting the basics running since a lot of the needed functions are not available out of the box (I2S support, boot from QPSI, LCD library support, etc). With the ESP32S3, use the 1.8 version IDE, not the 2.x series. The 2.x series does not yet support the addons that allow you to download files as littlefs into the QSPI flash. In short, audio works great on ESP32S3, really easy to use in Arduino IDE (while the STM32 is not easy to use in any IDE).
I also used the ESP8266Audio library, works great with the ESP32S3, I made some modifications just for tricorder applications that I can share. One to loop audio (handy for the tricorder sound), the other to play raw audio stored in C array. Eventually I'll submit those to author of ESP8266Audio.

 

[JaredTS486]

yes I did try at first to use a board with RP2040 processor. I also gave up. The I2S audio works great in circuitpython on RP2040, but I could not get it to work well in Ardunio. I tried several different audio libraries including modifying them. The problem is that the RP2040 does not have a real I2S port, it is emulated. Use the ESP32S3 instead it has a real I2S port, it works way better, more options and more memory available. On top of that dual core. I was using that it was working fine, until I decided I needed more, now with a STM32H750 processor because I needed more IO and more horsepower for the LCD. But I would really recommend the ESP32S3 really easy to use and get running, very well supported in Arduino.. The STM32 on the other hand I'm still working on getting the basics running since a lot of the needed functions are not available out of the box (I2S support, boot from QPSI, LCD library support, etc). With the ESP32S3, use the 1.8 version IDE, not the 2.x series. The 2.x series does not yet support the addons that allow you to download files as littlefs into the QSPI flash. In short, audio works great on ESP32S3, really easy to use in Arduino IDE (while the STM32 is not easy to use in any IDE).
I also used the ESP8266Audio library, works great with the ESP32S3, I made some modifications just for tricorder applications that I can share. One to loop audio (handy for the tricorder sound), the other to play raw audio stored in C array. Eventually I'll submit those to author of ESP8266Audio.

Do you recommend particular model ESP32-S3? The Adafruit FX Sound Board does not play loops as cleanly as I would like so it looks like I might pick one up.

 

[wystewart]

that would really depend on the features, number of IO, amount of memory that you need. some have battery charging and so forth. Right now I'm using the Expressif ESP32S3 C1 dev kit, but it is rather large physically and might not be a good choice to fit inside the tricorder body, but it works well for prototyping and code development on a breadboard as most of the IO are available. I'll check through my note on boards I looked at and suggest some if I can.

I would suggest a module with both PSRAM and QSPI flash. If you plan to store images and audio files, mininum 8MB flash, 16MB preferred. You can allocate part of the flash to store the program code and part of it to be used as a littlefs file system. There is a plugin for the Arduino IDE (version 1.8 only doesn't work with 2.x), that will collect files in a data folder in your sketch and create a littlefs file system in the QSPI flash. but you would also need a partition map, text file, in the sketch folder to add the littlefs partition and also optimize the flash usage (there are default paritions for redundant code and OTA updates that aren't really needed and just chew up space). If it helps I can provide the partition file I use. The PSRAM is handy if you want to allocate (malloc) arrays to store data, images, sound into memory.

With ESP32S3 it is also possible to use FREERtos and use both cores to say having one doing display and the other sound.

With my modified ESP8266Audio library, sound loops cleanly on the ESP32S3. I can also play MP3 on ESP32S3 (but I've only tried one file though - data life form song) Now if I could only get I2S audio working cleanly on my STM32H750, turning out to be a pain.

 

[Arky]

Adafruit Audio FX Mini Sound Board - WAV/OGG Trigger - 2MB Flash

This is what I used for the sound effects.

 

[JaredTS486]

Going to see if a TinyS3 will work for this. I tried everything I could with the Pico short of using CircuitPython and trying to re-create the TFT animation in python. Still no guarantee that will help.

I do have a Adafruit Mini FX soundboard that works fine and is easier to set up but there is a 100-200ms silence gap between loops and I want to try and save enough space to fit in a 2000mah battery.

Side note my Reed Switch keeps getting stuck closed and I have to tap it to get it open again. I am wondering if I damaged it or if the switching current is too high for it. When no power is being supplied it switches with the magnet correctly but eventually with power it gets stuck. With everything plugged in I am consuming ~0.45a @ 4.1v. I'm just ordering a new one to see if I have the same issue but if anyone has any thoughts let me know.

 

[wystewart]

The TinyS3 looks to be a good board. I had trouble with it in circuitpython with my phaser project, but did not try it in Arduino. I do have Arky's code up and running on the ESP32-S3 C1 dev board, the tinyS3 won't be much different than my implementation other than choice of IO pins, so let me know if you have any questions. I have ESP8266audio running also mixing audio.

You'd have to check the specs of the reed switch to know if the current is an issue, you should be below 50% of the max rated datasheet current. If you have a current spike on turn-on due to a lot of low ESR capacitance to charge for example, that can cause issues with reed switch reliability. The ESP32S3 has a really low power sleep mode, so another option is leave it always powered (although you may need to disconnect power from other external perpherials), and use the wake up function, where the reed switch triggers a wakeup via GPIO.

As I mentioned before, I'm going with an STM32H750 to be able drive a higher resolution 24bit RGB display. I finally got sound to work and booting code from QSPI. So much harder to do on STM32, so much easier on ESP32S3 with ardiuno ... Next step, connect the LCD and write a display library since none exist to support this display configuration... then to port ESP8266audio to the this....

 

[JaredTS486]

I ended up getting a couple of the reed switches below.

https://www.mouser.com/ProductDetail/876-KSK-1C90F-1015

https://www.mouser.com/ProductDetail/876-KSK-1C90U-2034

They have a .5a switching max and a 1a constant max. Both work well for switching the entire power supply. No issues so far.

I will probably get the TinyS3 to switch to a low power state since it seems that it is a little slower to boot than the Pico. I really want that tricorder opening noise to be as instantaneous as possible.

I also discovered that you could place another magnet next to the reed switch on the same plane without triggering it. I modified the fusion lower panel file to hold another magnet so it will hold closed.

Also if anyone has suggestions on helping the panel hinges be more conductive to reduce the flickering let me know. I am experimenting with some lubricants I have added some graphite / carbon fiber dust too.

 

[dkraude]

Love the research and design! Well done.

 

[wystewart]

I will probably get the TinyS3 to switch to a low power state since it seems that it is a little slower to boot than the Pico. I really want that tricorder opening noise to be as instantaneous as possible.

I'll check how quickly my ESP32S3 board gets the tricorder open noise to start. Thought it was pretty quick.

I also discovered that you could place another magnet next to the reed switch on the same plane without triggering it. I modified the fusion lower panel file to hold another magnet so it will hold closed.

Awesome idea for the magnet to hold it closed. Will implement that too in my design, thanks for the idea.

Also if anyone has suggestions on helping the panel hinges be more conductive to reduce the flickering let me know. I am experimenting with some lubricants I have added some graphite / carbon fiber dust too.

For the hinges, not sure what I'm going to do yet. I was leaning towards the brass hinges, then use a circuit to embed data over power so that two wires (PWR+ and GND) could be used instead of the 4 that I really need. However, hearing about your connection issue, flickering, and about Arky's report on the paint not sticking well, I'm leaning towards a 3D printed hinge with wires embedded. Maybe something like Mangy Dog's in https://www.youtube.com/watch?v=4kC63O7zNdg. But that will have to wait, until after I complete my board designs and the software.

 

[wystewart]

I will probably get the TinyS3 to switch to a low power state since it seems that it is a little slower to boot than the Pico. I really want that tricorder opening noise to be as instantaneous as possible.

my ESP32-S3 C1 dev board seems to start right away and do the open sound

 

[JaredTS486]

my ESP32-S3 C1 dev board seems to start right away and do the open sound

What pins did you use on your board to connect to the max98357a? Struggling to get it to work. Also did you find a decent way to get the TFT screen animation to work and loop the sounds using the ESP8266Audio library?

 

[wystewart]

what sort of problems are you having? here's my audio setup code.

out = new AudioOutputI2S();
out->SetPinout(13,12,14);

//int buffSizeSamples, AudioOutput *dest
mixer = new AudioOutputMixer(32, out);
stub[0] = mixer->NewInput();
stub[1] = mixer->NewInput();
stub[2] = mixer->NewInput();
stub[3] = mixer->NewInput();

I use mixer so that I can play beep on button press (using MPR121 touch pads), and periodically play alarm sound while playing the scanning sound. I did modify the ESP8266Audio library to add looping of the audio, and there's a good chance I messed with the I2S code a bit too. PM me your email, I can send you a zip of the modified library.

yes I do have audio, LCD, touch button and an LED strip running well at the same time without RTOS. This is my loop

void loop() {
currentMillis = millis();

SoundLoop();

if (millis()-start > 5000) {
if (!go) {
PlayAlertSound();
go = true;
}

if (millis()-start > 10000) {
go = false;
start = millis();
}
}

DM_FrameTop();
DM_FrameBottom();
DM_Button1();
DM_Button2();
DM_Button3();
DM_Bargraph1();
DM_Bargraph2();
DM_Bargraph3();
DM_Bargraph4();
DM_Bargraph5();
DM_Bargraph6();
DM_Bargraph7();
DM_Bargraph8();
DM_ScrollingText();

ButtonUpdate();

LEDLoop();
}

 

[JaredTS486]

I am trying to figure out what pins I need to connect to from the board above to the MAX98357A. I do not know how to tell if they need a special pin like I2C/SPI. Adafruit just says they can be "3.3-5v logic". The ILI9341 is using 34-37 plus 8 and 9 but I think I can move 8/9 to 6/7. I probably just need to sleep on this, having trouble finding anything on Google.

 

[wystewart]

View attachment 1713798
I am trying to figure out what pins I need to connect to from the board above to the MAX98357A. I do not know how to tell if they need a special pin like I2C/SPI. Adafruit just says they can be "3.3-5v logic". The ILI9341 is using 34-37 plus 8 and 9 but I think I can move 8/9 to 6/7. I probably just need to sleep on this, having trouble finding anything on Google.

I think there is a restriction as to where WS is compared to the other two pins (sequential order restrictions). Best to put them in sequential order, I will check that order on my config tonight. Pretty I had to mod the I2S code of ESP8266Audio to get things to work. I just modded a few files, so I should be able to attach them here tonight.

 

[wystewart]

try using a WS pin that is next to BCK pin. In the expressif doc, they show example with BCK at pin 4 and WS at pin 5. In my case I have BCK on pin 13 and WS on pin 12 and data on 14, but I don't think you have those pins available on the tinys3, but it does seem you have 4,5,6 or 3,4,5. Try those.

Also note that the max98357 is not supposed to be able to play 22050 audio according to datasheet, although it does seem to. Be sure that the sample rate of your audio files matches what the max98357 is capable of. It does have limitations.

The ESP8266 library as well as the underlying expressif I2S library seem to have had changes since I last compiled, so it is also possible that those changes have caused problems that I don't see.

not sure what exactly I modified in the ESP8266 library, but the files I modified are attached. In AudioGeneratorWAV certainly I added the ability to loop the audio file. The AudioGeneratorRAW file doesn't exist in the library, it is a file I added to play raw audio from a data array in a .h file.

Don't use IO0 since that also goes to the on board switch which will short IO0 if button is pressed.

 

[JaredTS486]

try using a WS pin that is next to BCK pin. In the expressif doc, they show example with BCK at pin 4 and WS at pin 5. In my case I have BCK on pin 13 and WS on pin 12 and data on 14, but I don't think you have those pins available on the tinys3, but it does seem you have 4,5,6 or 3,4,5. Try those.

Also note that the max98357 is not supposed to be able to play 22050 audio according to datasheet, although it does seem to. Be sure that the sample rate of your audio files matches what the max98357 is capable of. It does have limitations.

The ESP8266 library as well as the underlying expressif I2S library seem to have had changes since I last compiled, so it is also possible that those changes have caused problems that I don't see.

not sure what exactly I modified in the ESP8266 library, but the files I modified are attached. In AudioGeneratorWAV certainly I added the ability to loop the audio file. The AudioGeneratorRAW file doesn't exist in the library, it is a file I added to play raw audio from a data array in a .h file.

Don't use IO0 since that also goes to the on board switch which will short IO0 if button is pressed.

Thanks for the help. I managed to get it working with those pins. It looks like trying to use almost anything but WAVs from PROGMEM does not work on for me.

I got a basic loop going with a wav file I made up (see attached) pretty sure I got the quality settings right. But I am hearing a lot of white noise and a loud pop in between loops. Wondering if you had the same issues. If you are using a wav file as a PROGMEM / hex array can you send me a copy? I want to try and see if it is just my file.

Thanks again!

 

[wystewart]

all my sound files are in littlefs partition in Qspi flash. Attached is my scanning file. I have no pops, no white noise or anything beyond what is in the recorded file.

 

[wystewart]

I'll see if I can try my tinyS3 in place of the board I use and see if could be the tinys3. But give me sometime (few weeks) to get to it.

 

[wystewart]

JaredTS486 any progress resolving your white noise popping issues? I haven't had a chance to try the tinyS3 in my older setup.