1

My application runs on an ESP32-CAM. It takes pictures and stores to SPIFFS. Periodically SPIFFS is checked and if images are found: 1- The image is saved to a FTP server 2- The image is summited to a Web Service that returns some informations about the image 3- Values are saved to mySQL 4- The image is deleted from SPIFFS

To read the image from SPIFFS I use the following fragment of code:

String imageName   = file.name();  
imageBuffer = "";
while (file.available()){
   imageBuffer += char(file.read());
}

The FTP transfer works perfectly with the following command:

ftp.WriteData((uint8_t*)imageBuffer.c_str(),imageBuffer.length());

The service the image is submited for analizes works OK if I use the command:

webClient.write(fb->buf,fb->len);

But as it is not usable in the current implementation of my project I am trying to work with:

webClient.write((uint8_t*) imageBuffer.c_str(), imageBuffer.length());

I get no errors in return but it just does not work and the web service at the destination returns nothing.

What Am I doing wrong?

Paulo Borges
  • 95
  • 8
  • 1
    What you're doing is treating binary data as text. As soon as you get a 0 byte the string will be truncated. – Majenko Apr 07 '21 at 12:48
  • Your `webClient.write()` looks fine, but how are you framing this raw data? See [How does HTTP file upload work?](https://stackoverflow.com/questions/8659808). – Edgar Bonet Apr 07 '21 at 12:50
  • Short knowledge on how to handle this sort of data. "framing raw data"? What does it mean? – Paulo Borges Apr 07 '21 at 13:03
  • @Majenko: A `String` object does not truncate on the first NUL: it keeps all the bytes you push with `+=`, and the `length()` method returns the total number of bytes, including embedded NULs. – Edgar Bonet Apr 07 '21 at 13:04
  • Short answer: you can't store raw image data in a String. – Majenko Apr 07 '21 at 13:04
  • "What you're doing is treating binary data as text" Result of internet searchs. As it worked with FTP I tried the same sort or command with webClient but not working. To be honest I do not really understand what I am doing at this binary level. Searching the internet and trying to see what works. – Paulo Borges Apr 07 '21 at 13:05
  • @EdgarBonet I'd expect to end up with a string that has no NULLs in it, since adding a NULL to the end of a string would have no effect. – Majenko Apr 07 '21 at 13:05
  • “framing” is the stuff you put around the data. See the link I provided. @Majenko: yes, you can. – Edgar Bonet Apr 07 '21 at 13:05
  • I would like to understand the reason it works with FTP but not with webClient – Paulo Borges Apr 07 '21 at 13:06
  • My poor understanding tells me something to do with spaces and Nulls but do not know how to sort it out. – Paulo Borges Apr 07 '21 at 13:08
  • @EdgarBonet Internally String::concat() uses strcpy(). Ergo it will only copy anything up to the first NULL, replacing any end NULL that's already there. – Majenko Apr 07 '21 at 13:08
  • @Majenko: try it! I did. – Edgar Bonet Apr 07 '21 at 13:15
  • I will try it. Thanks – Paulo Borges Apr 07 '21 at 13:15
  • 1
    @EdgarBonet I'd rather not, thanks. I stay *well* away from String and anything to do with it. Never use String for *anything*. Not only is it generally a bad idea, you just can't trust what it's doing. And regardless: storing raw binary data in a string is asking for trouble. – Majenko Apr 07 '21 at 13:16

2 Answers2

1

You should not use String to store binary data.

Instead allocate yourself an array of uint8_t of the right size.

uint8_t *imageData;
size_t len = file.size();

imageData = alloca(len);
file.readBytes(imageData, len);

webClient.write(imageData, len);

With that said, without knowing what the rest of your program is doing I can only surmise that you're just blasting raw data at the web server and hoping it sticks. That's not how web data transfers work. You have to craft the upload page properly as multi-part mime form data. Read this for more information

Majenko
  • 103,876
  • 5
  • 75
  • 133
  • Hi, thanks, on one side I am submitting it to the FTP library and assume it will handle the flow. The submission to the web service is HTTP so I do not know, but when I use webClient.write(fb->buf,fb->len); it works so I am assuming it is handling the 24K or so bytes of the image. What do you suggest me to do? Thanks – Paulo Borges Apr 07 '21 at 16:49
1

OK, I got it to work as follows:

Not passing the binary anymore, passing the file name so the FTPPost function will do its own reading of the file on SPIFFS and will post in 256 bytes chunks. The same strategy works with HTTP post to the WebService.

  File picFile = SPIFFS.open(fileName,FILE_READ);
  const size_t bufferSize = 256;
  uint8_t buffer[bufferSize];
  while (picFile.available()) {
    size_t n = picFile.readBytes((char*) buffer, bufferSize);
    if (n != 0) {
      ftp.WriteData(buffer, n);
    } else {
      Serial.println(F("Buffer was empty"));
      break;
    }
  }
  picFile.close();
Paulo Borges
  • 95
  • 8