Upload files using POCO C++ Libraries

I have recently worked on uploading files using POCO and it gave me a hard time to get it work properly. I dint get much working tutorials on client side programming to upload a binary file to a upload link using POCO. So I thought of writing one by myself to help others who are facing the same problem.

Below is the code to upload any file as multipart form data by reading the entire data of file as binary :

Note that this may not work for large files as reading the entire data (1GB or more) in a variable will take a lot of memory and will slow down the system. You must consider chunked upload for this case by reading the file data in a fixed size buffer and uploading in chunks.

void uploadFile(const char* uploadLink, const char* filePath, const char* filename)
{
   try
   {
      //Prepare request
      Poco::URI uri(uploadLink);

      Poco::Net::HTTPClientSession session(uri.getHost(), uri.getPort());

      session.setKeepAlive(true);

      // prepare path
      std::string path(uri.getPathAndQuery());
      if (path.empty())
      {
         path = "/";
      }

      Poco::Net::HTTPRequest req(Poco::Net::HTTPRequest::HTTP_POST, path, Poco::Net::HTTPMessage::HTTP_1_1);

      // Set headers
      // Set Multipart type and some boundary value 
      req.setContentType("multipart/form-data; boundary=-------------------------87142694621188");

     // Append boundary, content type and disposition to file data

      std::string boundary = "-------------------------87142694621188";
      std::string data1("---------------------------87142694621188\r\nContent-Disposition: form-data; name=\"data\"; filename=\"");
      std::string data2(filename);
      std::string data3("\";\r\nContent-Type: application/octet-stream\r\n\r\n");

      std::string data4("\r\n---------------------------87142694621188--\r\n");
  
    // Read File Data in binary
      std::ifstream file (filePath,std::ios::binary);
      std::ostringstream ostrm;
      ostrm << file.rdbuf();

      std::string reqBody;
      reqBody.append(data1);
      reqBody.append(data2);
      reqBody.append(data3);
      reqBody.append(ostrm.str());
      reqBody.append(data4);

      req.setContentLength(reqBody.length());

      req.setKeepAlive(true);
      
      // sends request, returns open stream
      std::ostream& myOStream = session.sendRequest(req);
      // sends the body
      myOStream << reqBody;

      Poco::Net::HTTPResponse res;

      std::istream& rs = session.receiveResponse(res);

      //Get status code
      int statusCode = (int)res.getStatus();

      //Get status
      std::string status = res.getReason();

      std::string response;
      while(rs)
      {
         response.push_back(char(rs.get()));
      }
      std::cout<<"\nStatusCode: "<<statusCode<<"\nStatus: "<<status<<"\nResponse: "<<response;
  }
  catch(Poco::Exception& exception)
  {
       //Set Response for Exception
       std::cout<<"\nException occurred while uploading: "<<exception.displayText();
  }
}

The above method accepts the file name, upload link and complete file path (path along with file name) as parameters. Then we read the entire file data as binary and send the HTTP POST request to the server with the complete file data and boundaries appended. This uploads the file to server and we get the response received and the Http Status code.

Written By: Neha Gupta

Advertisements

QFileSystemWatcher: Monitor directories and get Rename, Delete, Add, Modify notifications with the name and path of the modified file/dir.

QFileSystemWatcher gives notification if the watched directory is modified. But it does not give any information about the type of change. So here I am going to explain that how you can find the change (rename, add, delete, modify) and the name of the file/dir (inside the watched dir) that is modified.

First of all add a class to your project. Lets name it “MyFileSystemWatcher”.

Now add a class variable of type QFileSystemWatcher to this class and a function to add paths to watch.

We will add two slots directoryUpdated(QString path) and fileUpdated(QString path) that will be connected to corresponding signals that are emitted when some change occurs in the watched dir/file.

Also we need a variable to maintain the list of current contents in a directory, and that can be compared to the directory contents after a change is notified to find out the name of file changed and the type of change. We’ll use a QMap having a  QString as key and QStringList as its value for this purpose. The key will be the path of some directory and the value will be the list of names of all files/directories it contains.

Hence moving on, your header file  MyFileSystemWatcher.h will look like:

#ifndef MYFILESYSTEMWATCHER_H

#define MYFILESYSTEMWATCHER_H

#include <QObject>
#include <QStringList>
#include <QMap>
#include <QFileSystemWatcher>

class MyFileSystemWatcher : public QObject
{
   Q_OBJECT

   public:

      static MyFileSystemWatcher* _instance; //singleton class instance

      QMap<QString, QStringList> _currContents; //maintain list of current contents of each watched directory
 
      QFileSystemWatcher* _sysWatcher;  //QFileSystemWatcher variable
     
      static void addWatchPath(QString path);

public slots:

      void directoryUpdated(const QString & path);  //called when a watched directory is updated. "path" is the path of watched directory in which change occured
      void fileUpdated(const QString & path);   //called when a watched file is modified."path" is the path of watched file which is modified
};

In the implementation file, we will allocate memory to the singleton instance of class and the QFileSystemWatcher variable “_sysWatcher”, also we will maintain the currents contents of each directory path that is watched(added to watcher) and compare the contents in the directoryUpdated() method called when a watched directory is modified.

So here is what the implementation file MyFileSystemWatcher.cpp will look like:

#include "MyFileSystemWatcher.h"

MyFileSystemWatcher* MyFileSystemWatcher::_instance = 0;

// Add a file or directory path to be watched

void MyFileSystemWatcher::addWatchPath(QString path)
{
   qDebug()<<"Add to watch: "<<path;
  
// Allocate memory to instance if not already allocated 

   if(_instance == 0)
   {
        _instance = new MyFileSystemWatcher;
        _instance->_sysWatcher = new QFileSystemWatcher();

// Connect the directoryChanged and fileChanged signals of QFileSystemWatcher to corresponding slots

        connect(_instance->_sysWatcher,SIGNAL(directoryChanged( QString )),_instance,SLOT(directoryUpdated(QString)));
        connect(_instance->_sysWatcher,SIGNAL(fileChanged( QString )),_instance,SLOT(fileUpdated(QString)));
   }
    
   _instance->_sysWatcher->addPath(path);  //add path to watch

// Save the list of current contents if the added path is a directory
 
    QFileInfo f(path);

    if(f.isDir())
    {
         const QDir dirw(path);
         _instance->_currContents[path] = dirw.entryList(QDir::NoDotAndDotDot | QDir::AllDirs | QDir::Files, QDir::DirsFirst);
    }   

}

// Slot invoked whenever any of the watched directory is updated (some file in the watched dir is added, deleted or renamed)

void MyFileSystemWatcher::directoryUpdated(const QString & path)
{
    qDebug()<<"Directory updated: "<<path;

// Compare the latest contents to saved contents for the dir updated to find out the difference(change) 
    
    QStringList currEntryList = _instance->_currContents[path];
    const QDir dir(path);

    QStringList newEntryList = dir.entryList(QDir::NoDotAndDotDot  | QDir::AllDirs | QDir::Files, QDir::DirsFirst);

    QSet<QString> newDirSet = QSet<QString>::fromList( newEntryList );
 
    QSet<QString> currentDirSet = QSet<QString>::fromList( currEntryList );

   // Files that have been added
    QSet<QString> newFiles = newDirSet - currentDirSet;
    QStringList newFile = newFiles.toList();

   // Files that have been removed
    QSet<QString> deletedFiles = currentDirSet - newDirSet;
    QStringList deleteFile = deletedFiles.toList();

   // Update the current set
    _instance->_currContents[path] = newEntryList;

   if(!newFile.isEmpty() && !deleteFile.isEmpty())
   {
       // File/Dir is renamed

       if(newFile.count() == 1 && deleteFile.count() == 1)
       {
           qDebug()<<"File Renamed from "<<newFile.first()<<" to "<<deleteFile.first();
       }
   }

   else
   {
        // New File/Dir Added to Dir
        if(!newFile.isEmpty())
        {

             qDebug()<<"New Files/Dirs added: "<<newFile;

             foreach(QString file, newFile)
             {
                     //Handle Operation on new files.....
             }
        }

        // File/Dir is deleted from Dir

        if(!deleteFile.isEmpty())
        {
            qDebug()<<"Files/Dirs deleted: "<<deleteFile;
            foreach(QString file, deleteFile)
            {
                // Handle operation of each deleted file....
            }
        }

    }

  }


// Slot invoked whenever the watched file is modified

void MyFileSystemWatcher::fileUpdated(const QString & path)
{
    QFileInfo file(path);

    QString path1 = file.absolutePath();

    QString name = file.fileName();

    qDebug()<<"The file "<<name<<" at path "<<path1<<" is updated";
}

This way you may add any no. of paths to the watcher by calling the static method “addWatchPath:” and the path can be either a file or directory.
If you want to watch a directory as well as all its sub directories, you need to add the path of parent directory and path of all its sub directories recursively to the watcher.

For Example:

If you want to watch a directory “ABC” in the C drive, just call the method

MyFileSystemwatcher::addWatchPath("C:/ABC");

This is all about QFileSystemWatcher. However there is a known bug in using it for recursive monitoring on Windows. To know more about this check out my other post.

Written By: Neha Gupta

Win32 File Watcher API to monitor Directory changes

Here I am going to explain how to use Win32 File Watcher API  to watch a directory as well as its sub directories for any change and find out the type of change.

You may use this C++ API to watch any directory for changes on Windows Platform in any C++ compatible application as I am using it in my Qt Application because of a bug in QFileSystemWatcher when watching a directory and its sub directories recursively.  To know more about QFileSystemWatcher bug click here.

We can use either ReadDirectoryChangesW or FindFirstChangeNotification to get the notification of any change in a directory or its subdirectories. But the added advantage of ReadDirectoryChangesW is that it gives the information of change and the name of the file that is changed. On the other hand  FindFirstChangeNotification only notifies that some change happened but no other information about the type of change or the name of file.

So lets start with How to implement  ReadDirectoryChangesW to watch a directory. Here is the code:

//include headers

#include <windows.h>
#include <Winbase.h>
#include <stdlib.h>
#include <stdio.h>
#include <tchar.h>
#define MAX_DIRS 25
 #define MAX_FILES 255
 #define MAX_BUFFER 4096

 extern "C" {
 WINBASEAPI BOOL WINAPI
 ReadDirectoryChangesW( HANDLE hDirectory,
 LPVOID lpBuffer, DWORD nBufferLength,
 BOOL bWatchSubtree, DWORD dwNotifyFilter,
 LPDWORD lpBytesReturned,
 LPOVERLAPPED lpOverlapped,
 LPOVERLAPPED_COMPLETION_ROUTINE lpCompletionRoutine
 );
 }

 // all purpose structure to contain directory information and provide
 // the input buffer that is filled with file change data

 typedef struct _DIRECTORY_INFO {
    HANDLE hDir;
    TCHAR lpszDirName[MAX_PATH];
    CHAR lpBuffer[MAX_BUFFER];
    DWORD dwBufLength;
    OVERLAPPED Overlapped;
 }DIRECTORY_INFO, *PDIRECTORY_INFO, *LPDIRECTORY_INFO;

 DIRECTORY_INFO DirInfo[MAX_DIRS];   // Buffer for all of the directories
 TCHAR FileList[MAX_FILES*MAX_PATH]; // Buffer for all of the files
 DWORD numDirs;
//Method to start watching a directory. Call it on a separate thread so it wont block the main thread.  

void WatchDirectory(LPCWSTR path)
{
   char buf[2048];
   DWORD nRet;
   BOOL result=TRUE;
   char filename[MAX_PATH];
   DirInfo[0].hDir = CreateFile (path, GENERIC_READ|FILE_LIST_DIRECTORY, 
                                 FILE_SHARE_READ|FILE_SHARE_WRITE|FILE_SHARE_DELETE,
                                 NULL, OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS|FILE_FLAG_OVERLAPPED,
                                 NULL);

   if(DirInfo[0].hDir == INVALID_HANDLE_VALUE)
   {
       return; //cannot open folder
   }
   
   lstrcpy( DirInfo[0].lpszDirName, path);
   OVERLAPPED PollingOverlap;
   
   FILE_NOTIFY_INFORMATION* pNotify;
   int offset;
   PollingOverlap.OffsetHigh = 0;
   PollingOverlap.hEvent = CreateEvent(NULL,TRUE,FALSE,NULL);
   while(result)
   {
       result = ReadDirectoryChangesW(
                  DirInfo[0].hDir,// handle to the directory to be watched
                  &buf,// pointer to the buffer to receive the read results
                  sizeof(buf),// length of lpBuffer
                  TRUE,// flag for monitoring directory or directory tree
                  FILE_NOTIFY_CHANGE_FILE_NAME |
                  FILE_NOTIFY_CHANGE_DIR_NAME |
                  FILE_NOTIFY_CHANGE_SIZE,
                //FILE_NOTIFY_CHANGE_LAST_WRITE |
                //FILE_NOTIFY_CHANGE_LAST_ACCESS |
                //FILE_NOTIFY_CHANGE_CREATION,
                &nRet,// number of bytes returned
                &PollingOverlap,// pointer to structure needed for overlapped I/O
                NULL);
                
       WaitForSingleObject(PollingOverlap.hEvent,INFINITE);
       offset = 0;
       int rename = 0;
       char oldName[260];
       char newName[260];
       do
       {
           pNotify = (FILE_NOTIFY_INFORMATION*)((char*)buf + offset);
           strcpy(filename, "");
           int filenamelen = WideCharToMultiByte(CP_ACP, 0, pNotify->FileName, pNotify->FileNameLength/2, filename, sizeof(filename), NULL, NULL);
           filename[pNotify->FileNameLength/2] = '';
           switch(pNotify->Action)
           {
               case FILE_ACTION_ADDED:
                   printf("\nThe file is added to the directory: [%s] \n", filename);
                   break;
               case FILE_ACTION_REMOVED:
                   printf("\nThe file is removed from the directory: [%s] \n", filename);
                   break;
               case FILE_ACTION_MODIFIED:
                   printf("\nThe file is modified. This can be a change in the time stamp or attributes: [%s]\n", filename);
                   break;
               case FILE_ACTION_RENAMED_OLD_NAME:
                   printf("\nThe file was renamed and this is the old name: [%s]\n", filename);
                   break;
               case FILE_ACTION_RENAMED_NEW_NAME:
                   printf("\nThe file was renamed and this is the new name: [%s]\n", filename);
                   break;
               default:
                   printf("\nDefault error.\n");
                   break;
            }

           offset += pNotify->NextEntryOffset;
       
        }while(pNotify->NextEntryOffset); //(offset != 0);
      }

    CloseHandle( DirInfo[0].hDir );

}

Hence this way you may read the changes in a directory and its sub directories (optional. Set 4th parameter to TRUE/FALSE in ReadDirectoryChangesW).

Written By: Neha Gupta