From 6f55b2eb80e8e1aea37165e24a257171342f797d Mon Sep 17 00:00:00 2001 From: Sebastian Romero Date: Thu, 4 Mar 2021 10:59:31 +0100 Subject: [PATCH 1/6] Change title --- content/tutorials/portenta-h7/vs-ard-gs/content.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/tutorials/portenta-h7/vs-ard-gs/content.md b/content/tutorials/portenta-h7/vs-ard-gs/content.md index eace8086..3821674f 100644 --- a/content/tutorials/portenta-h7/vs-ard-gs/content.md +++ b/content/tutorials/portenta-h7/vs-ard-gs/content.md @@ -1,4 +1,4 @@ -# Visualising the Vision Shield's Camera feed +# Getting Started With The Vision Shield Camera ## Overview This tutorial shows you how to capture frames from the Vision Shield Camera module and visualise the video output through a Processing sketch. From b302422a117ba6aac16cb731a09a1e5f3b4b80c2 Mon Sep 17 00:00:00 2001 From: Sebastian Romero Date: Thu, 4 Mar 2021 10:59:47 +0100 Subject: [PATCH 2/6] Fix broken image path --- content/tutorials/portenta-h7/vs-ard-gs/content.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/tutorials/portenta-h7/vs-ard-gs/content.md b/content/tutorials/portenta-h7/vs-ard-gs/content.md index 3821674f..472c7111 100644 --- a/content/tutorials/portenta-h7/vs-ard-gs/content.md +++ b/content/tutorials/portenta-h7/vs-ard-gs/content.md @@ -25,7 +25,7 @@ Connect the Vision Shield to your Portenta H7 as shown and plugin the H7 to your Open the board manager on the Arduino IDE and install the latest version of the MBED Core (1.3.2) -![](/Users/lenardgeorge/Documents/Arduino/03_Pro/arduino-pro-content/content/tutorials/portenta-h7/vs-ard-gs/assets/vs_ard_gs_core.svg) +![](assets/vs_ard_gs_core.svg) ### 2. Capturing the Frames From a30815e366157bad3f06bfde19eb41e8da4b0521 Mon Sep 17 00:00:00 2001 From: Sebastian Romero Date: Thu, 4 Mar 2021 11:17:37 +0100 Subject: [PATCH 3/6] Clean up formatting --- .../portenta-h7/vs-ard-gs/content.md | 37 +++++++------------ 1 file changed, 14 insertions(+), 23 deletions(-) diff --git a/content/tutorials/portenta-h7/vs-ard-gs/content.md b/content/tutorials/portenta-h7/vs-ard-gs/content.md index 472c7111..c00b9083 100644 --- a/content/tutorials/portenta-h7/vs-ard-gs/content.md +++ b/content/tutorials/portenta-h7/vs-ard-gs/content.md @@ -1,21 +1,20 @@ # Getting Started With The Vision Shield Camera ## Overview -This tutorial shows you how to capture frames from the Vision Shield Camera module and visualise the video output through a Processing sketch. +This tutorial shows you how to capture frames from the Vision Shield Camera module and visualise the video output through a Processing sketch. ### You Will Learn -- Capturing the frames from the camera. -- Sending the frames through Serial communication. -- Visualising the frames on Processing. +- Capturing the frames from the camera. +- Sending the frames through Serial communication. +- Visualising the frames on Processing. ### Required Hardware and Software -- Portenta H7 board () -- Portenta Vision Shield ( [LoRa](https://store.arduino.cc/portenta-vision-shield-lora) or [Ethernet](https://store.arduino.cc/portenta-vision-shield) ) -- USB C cable (either USB A to USB C or USB C to USB C) -- Arduino IDE 1.8.10+ or Arduino Pro IDE 0.0.4+ -- Processing 3.5.4 +- Portenta H7 board () +- Portenta Vision Shield ( [LoRa](https://store.arduino.cc/portenta-vision-shield-lora) or [Ethernet](https://store.arduino.cc/portenta-vision-shield) ) +- USB C cable (either USB A to USB C or USB C to USB C) +- Arduino IDE 1.8.10+ or Arduino Pro IDE 0.0.4+ +- Processing 3.5.4 ## Instructions - Accessing the vision shield is done through help of both Arduino and Processing tools. The Arduino sketch handles the capture of image data by the onboard camera while the java applet created on Processing helps us visualise this info. With the help of serial communication. you can create a channel that reroutes the frames from the Camera to the java applet. The following steps will run you through how to capture, package the data through the serial port and visualise the output in Processing. ### 1. The Basic Setup @@ -28,7 +27,6 @@ Open the board manager on the Arduino IDE and install the latest version of the ![](assets/vs_ard_gs_core.svg) ### 2. Capturing the Frames - To capture the frames you will need to use the `camera.h` thats included in the Mbed core. This library contains all apis related to frame capture,motion detection and pattern recognition. Include the header file in your sketch. ```c++ @@ -68,14 +66,13 @@ void loop() { ``` ### 3. Create the Processing Sketch - Open a new processing sketch file and name it `CameraCapture.pde`. ![Create a processing sketch]() Lets start by iniitialising the libraries and variables you will need to process the captured To process the data sent by the Vision Shield you will need to import the following libraries : -- `processing.serial.*` : a [Serial Library](https://processing.org/reference/libraries/serial/index.html) that is used to read and write data to external devices over the serial line. +- `processing.serial.*` : a [Serial Library](https://processing.org/reference/libraries/serial/index.html) that is used to read and write data to external devices over the serial line. - `java.nio.ByteBuffer` : a java class that provides access to operations on byte buffers ```java @@ -128,8 +125,7 @@ void draw() } ``` -### 4. Visualing the Frames - +### 4. Visualing the Frames For this step, you will need the help of the `serialEvent()` api to update the `myImage`as an when a new frame arrives on the serial port. ```java @@ -193,7 +189,6 @@ void serialEvent(Serial myPort) { ``` ### 5. Upload the sketch - Select the right serial port on your IDE and upload the Arduino sketch to your H7 and after a successful upload, run the `CameraViewer.pde` sketch. You should be able to see the rendered camera output on the processing canvas. ![Camera output on Processing]() @@ -201,11 +196,10 @@ Select the right serial port on your IDE and upload the Arduino sketch to your H ## Conclusion ### Next Steps -- A -- B +- A +- B ### Complete Sketch - The `CaptureRawBytes.ino` Sketch. ```c++ @@ -306,13 +300,10 @@ void serialEvent(Serial myPort) { } ``` - - ## Troubleshooting - ### Sketch Upload Troubleshooting Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. -**Authors:** Lenard George, YY +**Authors:** Lenard George, YY **Reviewed by:** ZZ [18.03.2020] **Last revision:** AA [27.3.2020] \ No newline at end of file From 11ab959a4694312f3119534722adf92626c4e85a Mon Sep 17 00:00:00 2001 From: Sebastian Romero Date: Thu, 4 Mar 2021 12:07:58 +0100 Subject: [PATCH 4/6] Fix typos and rephrase content up to chapter 4 --- .../portenta-h7/vs-ard-gs/content.md | 112 +++++++++++------- 1 file changed, 68 insertions(+), 44 deletions(-) diff --git a/content/tutorials/portenta-h7/vs-ard-gs/content.md b/content/tutorials/portenta-h7/vs-ard-gs/content.md index c00b9083..c2658123 100644 --- a/content/tutorials/portenta-h7/vs-ard-gs/content.md +++ b/content/tutorials/portenta-h7/vs-ard-gs/content.md @@ -4,45 +4,48 @@ This tutorial shows you how to capture frames from the Vision Shield Camera modu ### You Will Learn - Capturing the frames from the camera. -- Sending the frames through Serial communication. -- Visualising the frames on Processing. +- Sending the frames as a byte stream through a Serial connection. +- Visualising the frames in Processing. ### Required Hardware and Software -- Portenta H7 board () +- [Portenta H7 board](https://store.arduino.cc/portenta-h7) - Portenta Vision Shield ( [LoRa](https://store.arduino.cc/portenta-vision-shield-lora) or [Ethernet](https://store.arduino.cc/portenta-vision-shield) ) - USB C cable (either USB A to USB C or USB C to USB C) - Arduino IDE 1.8.10+ or Arduino Pro IDE 0.0.4+ -- Processing 3.5.4 +- Processing 3.5.4+ ## Instructions -Accessing the vision shield is done through help of both Arduino and Processing tools. The Arduino sketch handles the capture of image data by the onboard camera while the java applet created on Processing helps us visualise this info. With the help of serial communication. you can create a channel that reroutes the frames from the Camera to the java applet. The following steps will run you through how to capture, package the data through the serial port and visualise the output in Processing. +Accessing the Vision Shield's camera data is done with the help of both Arduino and the Processing IDE. The Arduino sketch handles the capture of image data by the on-board camera while the java applet created with Processing helps to visualise this data with the help of a serial connection. The following steps will run you through how to capture, package the data through the serial port and visualise the output in Processing. ### 1. The Basic Setup -Connect the Vision Shield to your Portenta H7 as shown and plugin the H7 to your computer. +Connect the Vision Shield to your Portenta H7 as shown and plug in the H7 to your computer. ![Connecting the Vision Shield to Portenta]() -Open the board manager on the Arduino IDE and install the latest version of the MBED Core (1.3.2) +Open the board manager in the Arduino IDE and install the latest version of the Portenta Core. ![](assets/vs_ard_gs_core.svg) ### 2. Capturing the Frames -To capture the frames you will need to use the `camera.h` thats included in the Mbed core. This library contains all apis related to frame capture,motion detection and pattern recognition. Include the header file in your sketch. -```c++ +Create a new Arduino sketch called `CameraCaptureRawBytes.ino` or just open the example sketch under Portenta_Camera->CameraCaptureRawBytes in the Arduino IDE. + +To capture the frames you will need to use the functions contained in `camera.h` which comes with the Portenta core. This library contains all APIs related to frame capturing, motion detection and pattern recognition. Include the header file in your sketch. + +```cpp #include "camera.h" ``` -Next, lets intialise a camera object and a frame buffer of the size 320*240 +Next, let's intialise a camera object and a frame buffer of the size 320*240 (76'800 bytes). -```c++ +```cpp CameraClass cam; uint8_t fb[320*240]; ``` -In the `setup()`, lets start the Serial communication at `921600` baud rate and iniitialise the camera using `cam.begin()`. +In the `setup()` function, let's start the Serial communication at `921600` baud rate and iniitialise the camera using `cam.begin()`. -```c++ +```cpp void setup() { Serial.begin(921600); // Init the cam QVGA, 30FPS @@ -50,19 +53,22 @@ void setup() { } ``` -In the loop we need to captire each Frame and send it over serial communicatio to the processing sketch that will display the frames. we will use the `grab(uint8_t *buffer, uint32_t timeout=5000);` api to fetch the frame from tthe frame buffer. +In the loop we need to capture each Frame and send it over a serial connection to the Processing sketch that will display the frames. We will use the `grab(uint8_t *buffer, uint32_t timeout=5000);` function to fetch the frame from the frame buffer and save it into our custom data buffer. -```c++ +```cpp void loop() { - // put your main code here, to run repeatedly: - if (Serial) { - // Grab frame and write to serial - if (cam.grab(fb) == 0) { - Serial.write(fb, 320*240); - } - while(Serial.read() != 1){}; + // put your main code here, to run repeatedly: + + // Wait until the receiver acknowledges + // that they are ready to receive new data + while(Serial.read() != 1){}; + + // Grab frame and write to serial + if (cam.grab(fb) == 0) { + Serial.write(fb, 320*240); } -} + +} ``` ### 3. Create the Processing Sketch @@ -70,7 +76,7 @@ Open a new processing sketch file and name it `CameraCapture.pde`. ![Create a processing sketch]() -Lets start by iniitialising the libraries and variables you will need to process the captured To process the data sent by the Vision Shield you will need to import the following libraries : +Let's start by importing the libraries and initialising the variables you will need to process the captured data. To process the data sent by the Vision Shield you will need to import the following libraries: - `processing.serial.*` : a [Serial Library](https://processing.org/reference/libraries/serial/index.html) that is used to read and write data to external devices over the serial line. - `java.nio.ByteBuffer` : a java class that provides access to operations on byte buffers @@ -80,7 +86,7 @@ import processing.serial.*; import java.nio.ByteBuffer; ``` -Next we initialise the following variables to process the recieved pixels from the serial line. We set the dimensions, pixel count, and bytes required per frame. +Next we initialise the following variables to process the received pixels from the serial port. We set the dimensions, pixel count, and bytes required per frame. ```java // must match resolution used in the sketch @@ -91,49 +97,67 @@ final int cameraPixelCount = cameraWidth * cameraHeight; final int bytesPerFrame = cameraWidth * cameraHeight * cameraBytesPerPixel; ``` -To recieve the frames you will need a Serial , PImage objects and an array to store the pixel values of the frame. Add the following variables to the code. +To recieve the frames you will need a Serial port, a PImage object and an array to store the pixel values of the frame. Add the following variables to the code. ```java Serial myPort; PImage myImage; byte[] frameBuffer = new byte[bytesPerFrame]; -int pixelPosition = 0; +int pixelPosition = 0; +int lastUpdate = 0; +boolean shouldRedraw = false; ``` -Here we will establish connection to the serial line, prepare the buffer to store the frame pixels and the image variables +Here we will establish a connection to the serial port and prepare the buffer to store the frame pixels. Additionally we send a byte to the Arduino sketch from Processing to let it know that it's ready to receive data. ```java -void setup() -{ - size(320,240); +void setup() { + size(640, 480); // if you know the serial port name - //myPort = new Serial(this, "COM5", 9600); // Windows + //myPort = new Serial(this, "COM5", 921600); // Windows //myPort = new Serial(this, "/dev/ttyACM0", 921600); // Linux - myPort = new Serial(this, "/dev/cu.usbmodem14101", 9600); // Mac + myPort = new Serial(this, "/dev/cu.usbmodem14101", 921600); // Mac // Set the number of bytes to buffer myPort.buffer(bytesPerFrame) // Create an image based on the camera's dimensions and format - myImage = createImage(cameraWidth, cameraHeight, ALPHA); -} + myImage = createImage(cameraWidth, cameraHeight, ALPHA); -void draw() -{ - image(myImage, 0,0); + // Let the Arduino sketch know we're ready to receive data + myPort.write(1); +} +``` + +The draw function checks if the connection is still alive and if there is any new data that can be drawn as an image. In that case the original image gets copied into a new image object so that it can be scaled up. + +```cpp +void draw() { + // Time out after 1.5 seconds and ask for new data + if(millis() - lastUpdate > 1500) { + println("Connection timed out."); + myPort.clear(); + myPort.write(1); + } + + if(shouldRedraw){ + PImage img = myImage.copy(); + img.resize(640, 480); + image(img, 0, 0); + shouldRedraw = false; + } } ``` ### 4. Visualing the Frames -For this step, you will need the help of the `serialEvent()` api to update the `myImage`as an when a new frame arrives on the serial port. +For this step, you will use the `serialEvent()` callback function to update the `myImage` when a new data is received on the serial port. ```java -void serialEvent(Serial myPort) { -} +void serialEvent(Serial myPort){} ``` -The first thing you need to inside this method is to read the bytes from the `frameBuffer` array which you can do with the help of the [`readBytes()`](https://processing.org/reference/libraries/serial/Serial_readBytes_.html) method that returns an integer value for the number of bytes read. +The first thing you need inside this method is to read the bytes from the `frameBuffer` array which you can do with the help of the [`readBytes()`](https://processing.org/reference/libraries/serial/Serial_readBytes_.html) method that returns the number of bytes read. ```java // Read the raw bytes @@ -153,7 +177,7 @@ for(int i=0; i < bytesRead; ++i){ Once all the pixels have been rendered, you need to send an acknowledgement back to the arduino sketch to send the pixels for the next frame. We will update the image through `updatePixels()`, reset the `pixelPosition` to `0`and write `1` to the serial port -``` c++ +``` cpp // Let the Arduino sketch know we received all pixels // and are ready for the next frame if(pixelPosition == cameraPixelCount){ @@ -202,7 +226,7 @@ Select the right serial port on your IDE and upload the Arduino sketch to your H ### Complete Sketch The `CaptureRawBytes.ino` Sketch. -```c++ +```cpp #include "camera.h" CameraClass cam; From 636a37cfc00c1b98c183efba481fb5a3ac17871e Mon Sep 17 00:00:00 2001 From: Sebastian Romero Date: Thu, 4 Mar 2021 12:50:34 +0100 Subject: [PATCH 5/6] Add explanations for updated sketch --- .../portenta-h7/vs-ard-gs/content.md | 203 +++++++++++------- 1 file changed, 125 insertions(+), 78 deletions(-) diff --git a/content/tutorials/portenta-h7/vs-ard-gs/content.md b/content/tutorials/portenta-h7/vs-ard-gs/content.md index c2658123..609037eb 100644 --- a/content/tutorials/portenta-h7/vs-ard-gs/content.md +++ b/content/tutorials/portenta-h7/vs-ard-gs/content.md @@ -132,7 +132,7 @@ void setup() { The draw function checks if the connection is still alive and if there is any new data that can be drawn as an image. In that case the original image gets copied into a new image object so that it can be scaled up. -```cpp +```java void draw() { // Time out after 1.5 seconds and ask for new data if(millis() - lastUpdate > 1500) { @@ -154,66 +154,81 @@ void draw() { For this step, you will use the `serialEvent()` callback function to update the `myImage` when a new data is received on the serial port. ```java -void serialEvent(Serial myPort){} +void serialEvent(Serial myPort) { + lastUpdate = millis(); + + // read the received bytes + myPort.readBytes(frameBuffer); + + // Access raw bytes via byte buffer + ByteBuffer bb = ByteBuffer.wrap(frameBuffer); + + int i = 0; + + while (bb.hasRemaining()) { + // read 8-bit pixel + byte pixelValue = bb.get(); + + // set pixel color + myImage.pixels[i++] = color(Byte.toUnsignedInt(pixelValue)); + } + + myImage.updatePixels(); + + // Ensures that the new image data is drawn in the next draw loop + shouldRedraw = true; + + // Let the Arduino sketch know we received all pixels + // and are ready for the next frame + myPort.write(1); +} ``` -The first thing you need inside this method is to read the bytes from the `frameBuffer` array which you can do with the help of the [`readBytes()`](https://processing.org/reference/libraries/serial/Serial_readBytes_.html) method that returns the number of bytes read. +The first thing we do inside this method is to update the timestamp for when the last data was read. This is to detect and recover from a connection timeout. Then read the bytes from the `frameBuffer` array which you can do with the help of the [`readBytes()`](https://processing.org/reference/libraries/serial/Serial_readBytes_.html) method that returns the number of bytes read. ```java -// Read the raw bytes -int bytesRead = myPort.readBytes(frameBuffer); +lastUpdate = millis(); + +// read the received bytes +myPort.readBytes(frameBuffer); +``` + +Then the frame buffer is translated into a ByteBuffer that allows for easy and safe access to the underlying bytes without having to worry about the array indices. + +```cpp +// Access raw bytes via byte buffer +ByteBuffer bb = ByteBuffer.wrap(frameBuffer); ``` -Next we parse the frame buffer to get the different pixel values ; draw the image based on the pixelPosition and [`color()`](https://processing.org/reference/color_.html) and [`Byte.toUnsignedInt()`](https://docs.oracle.com/javase/8/docs/api/java/lang/Byte.html). We then update the size of the array +Next we read the frame buffer and convert the bytes into pixel color values. The image gets constructed by sequentially filling the pixels array of the image. The conversion of the raw data is done wih [`color()`](https://processing.org/reference/color_.html) and [`Byte.toUnsignedInt()`](https://docs.oracle.com/javase/8/docs/api/java/lang/Byte.html). ```java -// Reading the buffer and plotting the pixels -for(int i=0; i < bytesRead; ++i){ - byte pixelValue = frameBuffer[i]; - myImage.pixels[pixelPosition] = color(Byte.toUnsignedInt(pixelValue)); - ++pixelPosition; - } +int i = 0; + +while (bb.hasRemaining()) { + // read 8-bit pixel + byte pixelValue = bb.get(); + + // set pixel color + myImage.pixels[i++] = color(Byte.toUnsignedInt(pixelValue)); +} ``` -Once all the pixels have been rendered, you need to send an acknowledgement back to the arduino sketch to send the pixels for the next frame. We will update the image through `updatePixels()`, reset the `pixelPosition` to `0`and write `1` to the serial port +Once all the pixels have been updated, you need to tell the sketch to redraw the image. Additionally we send an acknowledgement back to the arduino sketch to ask it to send the pixels for the next frame. We update the image with `updatePixels()` and write `1` to the serial port for the acknowledgement. ``` cpp -// Let the Arduino sketch know we received all pixels -// and are ready for the next frame -if(pixelPosition == cameraPixelCount){ - myImage.updatePixels(); - pixelPosition = 0; - myPort.write(1); - } -``` +myImage.updatePixels(); -heres the complete sketch for the `serialEvent()` function +// Ensures that the new image data is drawn in the next draw loop +shouldRedraw = true; -```java -void serialEvent(Serial myPort) { - - // read the raw bytes - int bytesRead = myPort.readBytes(frameBuffer); - - for(int i=0; i < bytesRead; ++i){ - byte pixelValue = frameBuffer[i]; - myImage.pixels[pixelPosition] = color(Byte.toUnsignedInt(pixelValue)); - ++pixelPosition; - } - - // Let the Arduino sketch know we received all pixels - // and are ready for the next frame - if(pixelPosition == cameraPixelCount){ - myImage.updatePixels(); - pixelPosition = 0; - myPort.write(1); - } - -} +// Let the Arduino sketch know we received all pixels +// and are ready for the next frame +myPort.write(1); ``` ### 5. Upload the sketch -Select the right serial port on your IDE and upload the Arduino sketch to your H7 and after a successful upload, run the `CameraViewer.pde` sketch. You should be able to see the rendered camera output on the processing canvas. +Select the right serial port on your IDE and upload the Arduino sketch to your H7. After a successful upload, run the `CameraViewer.pde` sketch in Processing. You should be able to see the rendered camera output on the Processing canvas. ![Camera output on Processing]() @@ -233,8 +248,7 @@ CameraClass cam; uint8_t fb[320*240]; void setup() { - - Serial.begin(921600); + Serial.begin(921600); // Init the cam QVGA, 30FPS cam.begin(CAMERA_R320x240, 30); @@ -242,13 +256,16 @@ void setup() { void loop() { // put your main code here, to run repeatedly: - if (Serial) { - // Grab frame and write to serial - if (cam.grab(fb) == 0) { - Serial.write(fb, 320*240); - } - while(Serial.read() != 1){}; + + // Wait until the receiver acknowledges + // that they are ready to receive new data + while(Serial.read() != 1){}; + + // Grab frame and write to serial + if (cam.grab(fb) == 0) { + Serial.write(fb, 320*240); } + } ``` @@ -274,53 +291,83 @@ final int cameraWidth = 320; final int cameraHeight = 240; final int cameraBytesPerPixel = 1; final int cameraPixelCount = cameraWidth * cameraHeight; -final int bytesPerFrame = cameraWidth * cameraHeight * cameraBytesPerPixel; +final int bytesPerFrame = cameraPixelCount * cameraBytesPerPixel; PImage myImage; byte[] frameBuffer = new byte[bytesPerFrame]; -int pixelPosition = 0; +int lastUpdate = 0; +boolean shouldRedraw = false; -void setup() -{ - //initialise the canvas - size(320, 240); +void setup() { + size(640, 480); // if you have only ONE serial port active - //myPort = new Serial(this, Serial.list()[0], 9600); // if you have only ONE serial port active + //myPort = new Serial(this, Serial.list()[0], 921600); // if you have only ONE serial port active // if you know the serial port name - //myPort = new Serial(this, "COM5", 9600); // Windows + //myPort = new Serial(this, "COM5", 921600); // Windows //myPort = new Serial(this, "/dev/ttyACM0", 921600); // Linux - myPort = new Serial(this, "/dev/cu.usbmodem14101", 9600); // Mac + myPort = new Serial(this, "/dev/cu.usbmodem14401", 921600); // Mac // wait for full frame of bytes myPort.buffer(bytesPerFrame); + myImage = createImage(cameraWidth, cameraHeight, ALPHA); + + // Let the Arduino sketch know we're ready to receive data + myPort.write(1); } -void draw() -{ - image(myImage, 0, 0); +void draw() { + // Time out after 1.5 seconds and ask for new data + if(millis() - lastUpdate > 1500) { + println("Connection timed out."); + myPort.clear(); + myPort.write(1); + } + + if(shouldRedraw){ + PImage img = myImage.copy(); + img.resize(640, 480); + image(img, 0, 0); + shouldRedraw = false; + } } void serialEvent(Serial myPort) { + lastUpdate = millis(); - // read the raw bytes - int bytesRead = myPort.readBytes(frameBuffer); + // read the received bytes + myPort.readBytes(frameBuffer); + + // Access raw bytes via byte buffer + ByteBuffer bb = ByteBuffer.wrap(frameBuffer); - for(int i=0; i < bytesRead; ++i){ - byte pixelValue = frameBuffer[i]; - myImage.pixels[pixelPosition] = color(Byte.toUnsignedInt(pixelValue)); - ++pixelPosition; - } + /* + Ensure proper endianness of the data for > 8 bit values. + When using > 8bit values uncomment the following line and + adjust the translation to the pixel color. + */ + //bb.order(ByteOrder.BIG_ENDIAN); + + int i = 0; + + while (bb.hasRemaining()) { + // read 8-bit pixel + byte pixelValue = bb.get(); + + // set pixel color + myImage.pixels[i++] = color(Byte.toUnsignedInt(pixelValue)); + } + + myImage.updatePixels(); + + // Ensures that the new image data is drawn in the next draw loop + shouldRedraw = true; // Let the Arduino sketch know we received all pixels // and are ready for the next frame - if(pixelPosition == cameraPixelCount){ - myImage.updatePixels(); - pixelPosition = 0; - myPort.write(1); - } + myPort.write(1); } ``` @@ -328,6 +375,6 @@ void serialEvent(Serial myPort) { ### Sketch Upload Troubleshooting Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa. -**Authors:** Lenard George, YY -**Reviewed by:** ZZ [18.03.2020] +**Authors:** Lenard George, Sebastian Romero +**Reviewed by:** Sebastian Romero [2021-03-04] **Last revision:** AA [27.3.2020] \ No newline at end of file From c76151c764b80042fe2eedd91783a831fe4928f2 Mon Sep 17 00:00:00 2001 From: Sebastian Romero Date: Thu, 4 Mar 2021 12:53:34 +0100 Subject: [PATCH 6/6] Add correct info to metadata file --- .../portenta-h7/vs-ard-gs/metadata.json | 21 +++++++------------ 1 file changed, 7 insertions(+), 14 deletions(-) diff --git a/content/tutorials/portenta-h7/vs-ard-gs/metadata.json b/content/tutorials/portenta-h7/vs-ard-gs/metadata.json index c1d709c5..5afafbec 100644 --- a/content/tutorials/portenta-h7/vs-ard-gs/metadata.json +++ b/content/tutorials/portenta-h7/vs-ard-gs/metadata.json @@ -1,20 +1,13 @@ -{ - "id":"por-ard-gs", - "index": 1, - "title":"Setting Up Portenta H7 For Arduino", - "slug":"por-ard-gs", - "authors":[ - "Lenard George", - "Sebastian Hunkeler" - ], +{ + "title":"Getting Started With The Vision Shield Camera", "coverImage":{ - "src":"assets/por_ard_gs_cover.svg?sanitize=true" + "src":"assets/vs_ard_gs_core.svg?sanitize=true" }, "tags":[ "Getting Started", - "IDE", - "Setup", - "Blink" + "Camera", + "Processing", + "Serial" ], - "abstract":"This tutorial teaches you how to set up the board, how to configure your computer and how to run the classic Arduino blink example to verify if the configuration was successful." + "abstract":"This tutorial shows you how to capture frames from the Vision Shield Camera module and visualise the video output through a Processing sketch." }