External SPI Flash accessible over MSC

[dunkmann00]

I was wondering if it is possible to connect an external SPI Flash module (I was looking at one that uses the Winbond W25Q64) and communicate with it in CircuitPython and over USB? In particular I was looking to do this with the ItsyBitsy M0 Express.

I am new to CircuitPython but have been trying to pour over the docs and repo to see what I can find. I found this issue #1764 which is similar, but it seems that it is still on the 'long term' milestones list.

I would gladly try to take this on and implement it myself but I think I need some pointers to get me going in the right direction.

Also, just to answer the question of why I am looking to do this, I want to use the ItsyBitsy to, among other things, play sounds. The 2MB onboard flash is somewhat limiting for the sound files I want to store. I would like to add an external flash chip where I would store the sound files and interact with it in the same way as the onboard chip. Based on everything I've looked at I would like to think it is possible, but I am not sure where in the CircuitPython code I should start to try to implement this feature.

[deshipu]

That is not currently possible. You can mount an external file system (easiest to do currently with an SD card) to be visible to your code, but it won't be accessible over USB to your computer.
I suppose the easiest way that would work out of the box is to use an SD card, and use an external SD card adapter to put the files on it.
You could also probably replace the flash chip on your ItsyBitsy with a bigger one (most of them have the same pin-out), but I'm not entirely sure what sizes are currently supported. You can see what flash chips are supported here: https://github.com/adafruit/circuitpython/blob/master/supervisor/shared/external_flash/devices.h

[dhalbert]

I would echo what @deshipu said, and also note that the NeoTrellis M4 and the PyPortal boards already come with an 8MB flash chip. But the SD card would be the easiest solution in the long run, and you'd be able to have many more sound files than would fit on 8MB.

[dunkmann00]

Thank you both for your quick responses!

I understand what you are saying, the SD card option did seem the most straight forward approach. I was trying to keep the price of the project down as much as possible and wanted to put it in an enclosure. For those reasons I was leaning away from an SD card. I also don't think I personally would be able to replace the flash chip on the ItsyBitsy since it is physically smaller than on some of the other express boards.

So even though this functionality does not exist and is not the easiest option for me, I still am curious what USB code in CircuitPython would need to be modified/added to accomplish this? Is this technically possible to do, or is there something about how CP is designed that would prohibit this?

[dhalbert]

The CIRCUITPY filesystem that is exposed via USB is a single FAT filesystem. The USB mass-storage mechanism ("MSC") exposes a block-oriented low-level view of a single disk (and therefore a single filesystem, or at best multiple filesystems on a single block-oriented device). USB MSC cannot present a unified view of all the mounted filesystems, unless it somehow pretended it was one disk, which would be very tricky.

It would be nice if the USB interface were file-oriented, instead of block-oriented. Then it would be a lot easier to present unified filesystems (and it would also solve some filesystem corruption issues). But the only such USB protocol readily available that does that is MTP (Media Transfer Protocol), used by MP3 players and the like. Apple in its infinite marketing wisdom has declined to support MTP because it would compete with its own proprietary music players.

[dunkmann00]

Okay, so it can't appear as one unified filesystem, but would it be possible to expose them as two separate disks? So it would appear as something like CIRCUITPY and CIRCUITPY2?

[dhalbert]

Okay, so it can't appear as one unified filesystem, but would it be possible to expose them as two separate disks? So it would appear as something like CIRCUITPY and CIRCUITPY2?

Yes, but then it needs to be another MSC device, using another USB endpoint (two endpoints are needed on SAMD21, due to some kind of bug). On SAMD21 we have 8 endpoints, but 1 is for control, 2 are for CDC, 2 are for MSC, 1 is for HID, and 2 are for MIDI, so we're full up. Other microcontrollers sometimes have less than 8.

[dunkmann00]

Gotcha, I didn't realize all of those endpoints are set even when not in use.

I have another idea for how I can at least "expand" my flash storage. If I connect the SPI Module (Winbond W25Q64 or W25Q128 if I want 16MB) to the board and then modify the Itsy Bitsy's mpconfigboard.h to use the external flash module rather than the onboard flash, I should be able to use the external module just like the onboard flash is normally used. In this case the onboard flash would not be used by CP but this is not a big deal.

[dhalbert]

Sounds good. I will close this for now, but feel free to ask further questions, in, say, discord or the forums.

[deshipu]

I wonder if now that we can free up USB endpoints in boot.py it becomes a little less impossible to have this?

[tannewt]

I wonder if now that we can free up USB endpoints in boot.py it becomes a little less impossible to have this?

This is not an issue with endpoint count. USB MSC can have multiple drives over the same endpoints by using a logical unit (LUN) index.

Let's keep this discussion to #3477 since it is still open.