Upload/Download large file through WCF REST Service streaming

Hello Frnds,

There was a requirement related to this and before I started working on this I thought everything would be simple and straight forward. But When I finished with the coding as per my thought, I had a problem regarding uploading a big file (>1 GB) and downloading the file and the problem was:
Problem #1: Getting
Problem #2: File was getting downloaded but browser was not able to open the file in specific file format because he din't have the information about "what type of file it is? (file mimetype)".
It took a little while to solve this problem for me and the fix was very simple. All I had to do is to return the file type (mimetype info) as part of response header and here was the trick, and I thought it could be good information to share with you all.
So Let's do it.

I'll start with WCF REST service definition and this is how it should be:

 [ServiceContract]  
   public interface IFileHandlerService  
   {  
     [OperationContract]  
     [XmlSerializerFormat]  
     [Description("Get the template for window")]  
     [WebInvoke(Method = "POST", UriTemplate = "file?fileName={fileName}")]  
     void UploadFile(string fileName, Stream fileContent);  

     [OperationContract]  
     [XmlSerializerFormat]  
     [Description("Get the template for window")]  
     [WebInvoke(Method = "GET", UriTemplate = "file?fileName={fileName}")]  
     Stream DownloadFile(string fileName);  
   }  

and the here is implementation for above service definition:

public class FileHandlerService : BaseFileHandlerService, IFileHandlerService
    {
        public Stream DownloadFile(string fileName)
        {
            Stream fileStream = GetFileStream(fileName); //Here I am getting the actual file as a stream. You could get the file stream from
            //many sources like i.e. from ftp, UNC shared path, local drive itself etc.
            fileStream.Seek(0, SeekOrigin.Begin);

            //Below three lines are very important as it will have the information about the file for browser.
            //without this information browser won't be able to open the file properly and this is what I was I talking about.
            String headerInfo = "attachment; filename=" + fileName;
            WebOperationContext.Current.OutgoingResponse.Headers["Content-Disposition"] = headerInfo;
            WebOperationContext.Current.OutgoingResponse.ContentType = MimeMapping.GetMimeMapping(fileName);

            return fileStream;
        }


        public void UploadFile(string fileName, Stream fileContent)
        {
           // Here you can have your own implementation to save file in different location i.e. FTP Server using ftp, 
           //shared server through UNC path or in the same server etc.
           UploadFileToLocation(fileName, fileContent)
        }
      
    }

This is about the service implementation and here is endpoint and behavior configuration for web.config I have. All important and crucial configurations settings are highlighted. It's difficult to explain all those settings one by one so I would recommend to go through MSDN and read about it if required.

  <system.web>

    <compilation debug="true" targetFramework="4.5.2" />

    <httpRuntime targetFramework="4.5.2" maxRequestLength="2147483647" executionTimeout= "3600" />

    <httpModules>

      <add name="ApplicationInsightsWebTracking" type="Microsoft.ApplicationInsights.Web.ApplicationInsightsHttpModule, Microsoft.AI.Web" />

    </httpModules>

  </system.web>  

<system.serviceModel>

    <services>

      <service name="P21.FileHandlingService.FileHandlerService">

        <endpoint address="" binding="webHttpBinding" behaviorConfiguration="restfulBehavior" 

                  contract="P21.FileHandlingService.IFileHandlerService" bindingConfiguration="WebHttpBindingConfig" />

        <host>

          <baseAddresses>

            <add baseAddress="http://localhost:81/Service" />

          </baseAddresses>

        </host>

        <endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange" />

       </service>

    </services>

    <bindings>

      <webHttpBinding>

        <binding name="WebHttpBindingConfig" openTimeout="0:25:00" closeTimeout="0:25:00" sendTimeout="0:25:00" receiveTimeout="0:25:00" 

                 maxBufferPoolSize="2147483647" maxBufferSize="2147483647" maxReceivedMessageSize="5368709120" transferMode="Streamed">

          <readerQuotas maxDepth="2147483647"

                        maxStringContentLength="2147483647"

                        maxArrayLength="2147483647"

                        maxBytesPerRead="2147483647"

                        maxNameTableCharCount="2147483647"/>

          <security mode="None" />

        </binding>

      </webHttpBinding>     

    </bindings>

    <behaviors>

      <endpointBehaviors>

        <behavior name="restfulBehavior">

          <webHttp/>

          <dispatcherSynchronization asynchronousSendEnabled="true"/>

        </behavior>

      </endpointBehaviors>

      <serviceBehaviors>

        <behavior>

          <serviceMetadata httpGetEnabled="true" httpsGetEnabled="true" />

          <serviceDebug includeExceptionDetailInFaults="true" />

        </behavior>

      </serviceBehaviors>

    </behaviors>  

    <serviceHostingEnvironment aspNetCompatibilityEnabled="true" multipleSiteBindingsEnabled="true" />

  </system.serviceModel>

  <system.webServer>

    <security>

      <requestFiltering>

        <!--IIS Settings-->

        <requestLimits maxAllowedContentLength="2147483648" />

      </requestFiltering>

    </security>

    <modules runAllManagedModulesForAllRequests="true">

      <remove name="ApplicationInsightsWebTracking" />

      <add name="ApplicationInsightsWebTracking" type="Microsoft.ApplicationInsights.Web.ApplicationInsightsHttpModule, Microsoft.AI.Web" preCondition="managedHandler" />

    </modules>

    <!--

        To browse web app root directory during debugging, set the value below to true.

        Set to false before deployment to avoid disclosing web app folder information.

      -->

    <directoryBrowse enabled="true" />

    <validation validateIntegratedModeConfiguration="false" />

  </system.webServer>   

These settings you can change/modify based on your needs, but you should know two important things here:
1. httpRuntime maxRequestLength : this is the setting which tells what's the maximum request length to send data. i.e. If I need to send data in chunks of 10MB file then it must set to >10MB. Hence all depends on chunks data size.
2. requestLimits maxAllowedContentLength: this the same settings at IIS server level.
3. binding maxReceivedMessageSize: This must be the maximum possible values of you file size which you want to upload.

Hence these values must be set accordingly.

You can test this service using jquery ajax call or C#.

You can consume this service through jquery ajax call or C#. Here is an example for jquery. But below jquery code doesn't have logic to chunking data. Hence this will fail if you want to send large file (i.e. >30 MB).

<!DOCTYPE html>
<html>
<head>
    <title></title>
    <meta charset="UTF-8" />
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script>
    <script type="text/javascript">
        function UploadFile() {
            // grab your file object from a file input
            fileData = document.getElementById("fileUpload").files[0];

            $.ajax({
                url: 'http://localhost:81/Service1.svc/file?fileName=' + fileData.name,
                type: 'POST',
                data: fileData,
                cache: false,
                dataType: 'json',
                processData: false, // Don't process the files
                contentType: false, // Set content type to false as jQuery will tell the server its a query string request
                success: function (result) {
                    alert('successful..');
                },
                error: function (result) {
                    if (result.statusText === 'OK')
                        alert('successful..');
                    else
                        alert('Failed with reason: ' + result.statusText);
                }
            });

        }

        function DownloadFile(fileName, fileType) {

            var url = 'http://localhost:81/Service1.svc/file?fileName=' + fileName;
            window.location(url);             
        }

        function ListFiles()
        {
            $('#tblfiles').append('');
            $.ajax({
                url: 'http://localhost:81/Service1.svc/files', //wcf rest service which return files details.
                type: 'GET',
                cache: false,
                dataType: 'xml',
                processData: false, // Don't process the files
                contentType: false, // Set content type to false as jQuery will tell the server its a query string request
                success: function (result) {
                    var trHTML = '';
                    var i;
                    var x = result.getElementsByTagName("FileDetails");
                    for (i = 0; i < x.length; i++) {
                        var filename = x[i].childNodes[0].textContent;
                        var filesize = x[i].childNodes[2].textContent;
                        var fileType = x[i].childNodes[1].textContent;
                        trHTML += '<tr><td>' + filename + '</td><td>' + filesize + '</td><td>' + '<a href="#" onclick= "return DownloadFile(\'' + filename + '\',\'' + fileType + '\');">Download</a>' + '</td></tr>';
                    }
                    $('#tblfiles').append(trHTML);
                },
                error: function (result) {
                    alert('Failed with reason: ' + result.statusText);
                }
            });
        }
    </script>
</head>
<body>
    <div>
        <div>
            <input type="file" id="fileUpload" value="" />
            &nbsp;&nbsp;
            <button id="btnUpload" onclick="UploadFile()">
                Upload
            </button>
            <br/>
            <button id="btnList" onclick="ListFiles()">
                List Files
            </button>
        </div>
        <br/>
        <div id="filesdiv">
            <table id="tblfiles" border='1'>
                <tr>
                    <th>FileName</th>
                    <th>Size</th>
                    <th>#</th>
                    <!--<th>#</th>-->
                </tr>
            </table>
        </div>
    </div>
</body>
</html>

Note: You have to chunk the data if it is a large file. In this case above jquery code won't work as it tries to send whole data at one shot.

This is the rendered UI for above html.

What I have not shown in above service definition is "rest service method for getting the file details.". This is not a big deal and can be done easily in the way you want.

Here is the C# example which calls this service to upload a large video file (>1GB) and send data in chunks.

try
            {
                string fileName = "The_Angry_Birds_Movie_2016.mkv";
                HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create("http://localhost:81/Service1.svc/file?fileName=" + fileName);
                Stream fileStream = File.OpenRead(@"C:\Personal\Movies\" + fileName);
                HttpWebResponse resp;

                req.Method = "POST";
                req.SendChunked = true;
                req.AllowWriteStreamBuffering = false;
                req.KeepAlive = true;
                req.Timeout = int.MaxValue; //this I did for safe side but it shouldn't be case in production.
                req.ContentType = MimeMapping.GetMimeMapping(fileName);
                // The post message header
                string strFileFormName = "file";
                string strBoundary = "———-" + DateTime.Now.Ticks.ToString("x");
                StringBuilder sb = new StringBuilder();
                sb.Append("–");
                sb.Append(strBoundary);
                sb.Append("\r\n");
                sb.Append("Content - Disposition: form - data; name =\"");
                sb.Append(strFileFormName);
                sb.Append("\"; filename =\"");
                sb.Append(fileName);
                sb.Append("\"");
                sb.Append("\r\n");
                sb.Append("Content - Type: ");
                sb.Append("application / octet - stream");
                sb.Append("\r\n");
                sb.Append("\r\n");

                string strPostHeader = sb.ToString();
                byte[] postHeaderBytes = Encoding.UTF8.GetBytes(strPostHeader);
                long length = postHeaderBytes.Length + fileStream.Length;

                req.ContentLength = length;
                req.AllowWriteStreamBuffering = false;

                
                Stream reqStream = req.GetRequestStream();
                
                // Write the post header
                reqStream.Write(postHeaderBytes, 0, postHeaderBytes.Length);

                // Stream the file contents in small pieces (4096 bytes, max).

                byte[] buffer = new Byte[checked((uint)Math.Min(10485760, (int)fileStream.Length))]; //10 MB
                int bytesRead = 0;
                int count = 0;
                while ((bytesRead = fileStream.Read(buffer, 0, buffer.Length)) != 0)
                {
                    count += 10;
                    reqStream.Write(buffer, 0, bytesRead);
                    Console.WriteLine("DataWritten:" + count + "MB");
                }
                fileStream.Close();
                reqStream.Close();
                resp = (HttpWebResponse)req.GetResponse();
                
                Console.WriteLine(resp.StatusCode + ":"+ resp.StatusDescription);
            }
            catch(WebException ex)
            {
                Console.WriteLine(ex.Message);
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
            }
            finally
            {
                Console.ReadLine(); 
            }

Note: Above will handle upload/download for any type of files.

That's all. Thanks for reading. Feel free to provide your valuable feedback.