Search This Blog

Monday, May 18, 2015

Clicking a button to download a server file sent as a FileContentResult via ASP.NET Web Api

Scenario:
I have a Web Api controller that returns a .csv file as a FileContentResult.
In the UI, I have a <button> element that when clicked, I want it to trigger the file download to the local computer.

Solution:
After playing a bit with ajax calls and trying to use an anchor element instead a button to follow what many web posts suggest - which is to add "data" and "chartset" attributes plus the server uri that returns the content, like

'data:text/csv;charset=UTF-8,' + encodeURI(...)
and having no success, I then switched to a different approach - also vastly suggested on the web - to simply have the browser's window.location.href attribute set to the server uri that returns the file, like


window.location.href= encodeURI(...);
The simplest and neatest solution for this problem.

Tuesday, May 12, 2015

Http Request To Upload File(s) To The Server In ASP.NET Web Api

Instructions: if you need to attach files to a http request to upload them to a server, here's basically what you need to do:

  1. In your api post method, you need to verify that the request contains files, via Request.Content.IsMimeMultipartContent
  2. Go through the files attached to the request, using HttpContext.Request.Files, casting them to HttpPostedFileBase.
  3. Now you do what you want with the files.
To test this using a http client like google's Advanced REST Client, simply mark the request as a post method, and you will be able to click a "Files" option, like the in the image below:


Usage:
            try
            {
                if (Request.Content.IsMimeMultipartContent())
                {
                    HttpFileCollectionBase files = this.UmbracoContext.HttpContext.Request.Files;
                    foreach (String uploadedFileName in files)
                    {
                        HttpPostedFileBase httpPostedFileBase = files[uploadedFileName] as HttpPostedFileBase;

                        /* do what you want with the file, here */
                    }
                }
            }
            catch (Exception ex)
            {
                throw ex;
            }

Thursday, May 7, 2015

Use of typehead.js (autocomplete)

usage: GenerateAutoComplete(selectors.BankName, bankNamesArray); 

var GenerateAutoComplete = function (elementID, data, clear) {
    /* ------ Regular Typeahead autocomplete -------- */
    //var substringMatcher = function (strs) {
    //    return function findMatches(q, cb) {
    //        var matches, substrRegex;

    //        // an array that will be populated with substring matches
    //        matches = [];

    //        // regex used to determine if a string contains the substring `q`
    //        substrRegex = new RegExp(q, 'i');

    //        // iterate through the pool of strings and for any string that
    //        // contains the substring `q`, add it to the `matches` array
    //        $.each(strs, function (i, str) {
    //            if (substrRegex.test(str)) {
    //                // the typeahead jQuery plugin expects suggestions to a
    //                // JavaScript object, refer to typeahead docs for more info
    //                matches.push({ value: str });
    //            }
    //        });

    //        cb(matches);
    //    };
    //};

    //$(elementID).typeahead({
    //    hint: true,
    //    highlight: true,
    //    minLength: 1
    //},
    //{
    //    name: 'data',
    //    displayKey: 'value',
    //    source: substringMatcher(data)
    //});

    /*--------- Bloodhound Search Engine-----------*/
    clear = typeof clear === "undefined" ? true : clear;
    $(elementID).typeahead('destroy');

    if (clear) {
        $(elementID).val('');
    }

    // constructs the suggestion engine
    var bloodHoundData = new Bloodhound({
        datumTokenizer: Bloodhound.tokenizers.obj.whitespace('value'),
        queryTokenizer: Bloodhound.tokenizers.whitespace,
        local: $.map(data, function (ddlElement) { return { value: ddlElement }; }),
        limit: 10
    });

    // kicks off the loading/processing of `local` and `prefetch`
    bloodHoundData.initialize();

    $(elementID).typeahead({
        hint: true,
        highlight: true,
        minLength: 1
    },
    {
        name: 'data',
        displayKey: 'value',
        // `ttAdapter` wraps the suggestion engine in an adapter that
        // is compatible with the typeahead jQuery plugin
        source: bloodHoundData.ttAdapter()
    });
}

Categorize StackOverflow questions by tags

Follows a demo of a my "extension" to StackOverflow.
Here I can categorize all the questions I asked by tags. A click in the question will take you into the question page at StackOverflow.

Important note: this example works only as long as my local host is serving my proxy server. I have not found any .NET free web hosting services. My local host will only be up when I am using my computer, so chances are very high you will not see this working. But it does :-)

Remark: my scenario does not take into account paging in scraping the data. My questions page is one page only.

Search questions by tag:



Basically, what was done was: 
  1. Use server-side code as a proxy server, to be able to overcome the Same Origin Policy. One reason for the existence of this policy is described here.
  2. I am using .NET C#, so I made use of HtmlAgilityPack and ScrapySharp nuget packages to scrape the html from my StackOverflow questions page. There I extract all data I want to manipulate in the client, and send it back as JSON.
  3. Finally, in the client, you just do whatever you want with that data. I wanted to display an autocomplete textbox with all the tags I ever asked a question about, so this is what I have done - using typeahead.js for the autocomplete. For opening the pop-up I used jQueryUI.

I was quite happy after performing my first CORS successfully :-). I tried it a couple of times in the past, without success. Next step is doing so in the client side only, with no server code (this was my original intent, however I just came across stackoverflow material that lead me to the proxy-server solution, which is quite simple.)

The client source:

<script>
    $(document).ready(function() {
        function generateAutoComplete(elementID, data, clear) {
            /*--------- Bloodhound Search Engine-----------*/
            clear = typeof clear === "undefined" ? true : clear;
            $(elementID).typeahead('destroy');

            if (clear) {
                $(elementID).val('');
            }

            // constructs the suggestion engine
            var bloodHoundData = new Bloodhound({
                datumTokenizer: Bloodhound.tokenizers.obj.whitespace('value'),
                queryTokenizer: Bloodhound.tokenizers.whitespace,
                local: $.map(data, function(ddlElement) {
                    return {
                        value: ddlElement
                    };
                }),
                limit: 10
            });

            // kicks off the loading/processing of `local` and `prefetch`
            bloodHoundData.initialize();

            $(elementID).typeahead({
                hint: true,
                highlight: true,
                minLength: 1
            }, {
                name: 'data',
                displayKey: 'value',
                // `ttAdapter` wraps the suggestion engine in an adapter that
                // is compatible with the typeahead jQuery plugin
                source: bloodHoundData.ttAdapter()
            });
        }

        var serverData;
        $.ajax({
            url: 'http://localhost:56212/api/GetQuestions',
            method: 'get',
            success: function(data) {
                serverData = data;
                generateAutoComplete('#txtSearch', data.autocompleteSrc);
            },
            error: function(data) {
                console.log(data);
            },
            failed: function(data) {
                console.log(data);
            }
        });

        $("#btnShow").click(function() {
            var dialogContent = "<ul>";
            for (var i = 0; i < serverData.questionsPerTag.length; i++) {
                if (serverData.questionsPerTag[i].tag == $("#txtSearch").val()) {
                    for (var k = 0; k < serverData.questionsPerTag[i].questions.length; k++) {
                        dialogContent += "<li> <a class='link' href='" + serverData.questionsPerTag[i].links[k] + "'>" + serverData.questionsPerTag[i].questions[k] + " </a> </li><br />";
                    }
                    //dialogContent = serverData.questionsPerTag[i].questions.join();
                    break;
                }

            }
            dialogContent += "</ul>";

            $("#dialog").html(dialogContent);
            $("#dialog").dialog({
                height: 500,
                width: 1000
            });

        });

        $("#dialog").on("click", "li .link", function(event) {
            window.open(event.currentTarget.getAttribute("href"), "_blank");
        });

        $(".tt-dataset.tt-dataset-data").css("color", "000");
    });
</script>

Server-side code:
  public class ScrapingController : ApiController
    {
        [AllowAnonymous]
        [HttpGet]
        [Route("api/GetQuestions")]
        public dynamic GetQuestions()
        {

            //HttpContext.Current.Response.Headers.Add("Access-Control-Allow-Origin", "http://devrecipeshb.blogspot.co.il/*");
            HttpContext.Current.Response.Headers.Add("Access-Control-Allow-Origin", "*");
            HttpClient httpClient = new HttpClient();
            String result = httpClient.GetStringAsync("http://stackoverflow.com/users/1219280/veverke?tab=questions").Result;

            HtmlDocument doc = new HtmlDocument();
            doc.LoadHtml(result);
            HtmlNode divUserQuestions = doc.GetElementbyId("user-tab-questions");

            List questionsSummary = divUserQuestions.CssSelect(".question-summary .summary").ToList();
            List originalData = new List();

            foreach (HtmlNode questionSummary in questionsSummary)
            {
                originalData.Add(new SOQuestion
                {
                    Text = questionSummary.CssSelect("h3").FirstOrDefault().InnerText,
                    Link = questionsSummary.CssSelect(".question-hyperlink").FirstOrDefault().GetAttributeValue("href"),
                    Tags = ExtractTags(questionSummary.CssSelect(".tags").FirstOrDefault())
                });
            }

            List tags = new List(originalData.SelectMany(q => q.Tags).Distinct());
            List questionsPerTagResult = new List();

            foreach(string tag in tags)
            {
                SOQuestionsPerTag questionsPerTag = new SOQuestionsPerTag();

                questionsPerTag.Tag = tag;
                foreach(SOQuestion question in originalData)
                {
                    if (question.Tags != null && question.Tags.Contains(tag))
                    {
                        questionsPerTag.Questions.Add(question.Text);
                        questionsPerTag.Links.Add("http://www.stackoverflow.com/" + question.Link);
                    }
                }

                questionsPerTagResult.Add(questionsPerTag);
            }

            return new { autocompleteSrc = tags, questionsPerTag = questionsPerTagResult };

        }

        private List ExtractTags(HtmlNode tagsDiv)
        {
            return tagsDiv.InnerText.Trim().Split().ToList();
        }
    }

Wednesday, May 6, 2015

Localization in MVC .NET using C#

Refer to this post of mine at stackoverflow.

I spent hours trying to understand why my set up was not working, to figure out that each language that is found in the header MUST have a matching .resx file named [YouResourceFile].[language-2-digit-code].resx, otherwise localizing to that language will use the default resource, since the matching resource was not found.

Tuesday, May 5, 2015

Aggreagating rows in oracle per group using LISTAGG (version 11 and on)

protected T GetExistingImprovementFullData(string policyOrAccountNo, bool isLifeTrack, string usf, int taskNumber, string taskSpecificTableName, bool discardAlreadyApproved = false) where T : OraBaseImprovement, new()
        {
            T completeExistingImprovementData = new T();

            string query = string.Format(@"
                        SELECT *
                        FROM
                           (SELECT imp.improvement_id,
                                   imp.record_status,
                                   imp.user_comments,
                                   imp.usf_number,
                                   imp.task_number,
                                   imp.policy_number,
                                   imp.provident_fund_account_number,
                                   imp.improver_status,
                                   imp.improver_sub_status,
                                   imp.manager_batchly_approved,
                                   imp.manager_manually_approved,
                                   imp.manager_approval_date,
                                   imp.usf_manager,
                                   imp.create_date,
                                   imp.update_date,
                                   imp.tv08_status,
                                   LISTAGG('[' || ADAdocs.document_id        || ','
                                               || TRIM(ADAdocs.Doc_Type)|| ','
                                               || TO_CHAR(ADAdocs.Attachment_Date, 'dd/mm/yyyy hh24:mi:ss')    || ']',
                                           ',')
                            WITHIN
                             GROUP(ORDER BY impDocs.Document_Id) as documents
                            FROM pia_improvements imp
                            LEFT JOIN pia_improvement_attachments impDocs ON impDocs.improvement_id = imp.improvement_id
                            LEFT JOIN pia_ada_attached_documents ADAdocs ON ADAdocs.document_id = impDocs.document_id
                            WHERE {0} = '{1}'
                            AND imp.usf_number = '{2}'
                            AND imp.task_number = {3}
                            AND imp.record_status = 1   /* improvement is active */
                            {4}
                            GROUP BY imp.improvement_id,
                                     imp.record_status,
                                     imp.user_comments,
                                     imp.usf_number,
                                     imp.task_number,
                                     imp.policy_number,
                                     imp.provident_fund_account_number,
                                     imp.improver_status,
                                     imp.improver_sub_status,
                                     imp.manager_batchly_approved,
                                     imp.manager_manually_approved,
                                     imp.manager_approval_date,
                                     imp.usf_manager,
                                     imp.create_date,
                                     imp.update_date,
                                     imp.tv08_status ) baseImp
                            JOIN {5} task ON task.improvement_id = baseImp.improvement_id ",
                    DetermineWhereColumnNameBasedOnTrack(isLifeTrack),
                    policyOrAccountNo,
                    usf,
                    taskNumber,
                    discardAlreadyApproved ? GetDiscardImprovementAlreadyApprovedSQL() : string.Empty,
                    taskSpecificTableName);

            DataTable improvementDataTable = ExecuteDataTable(query);

            if (improvementDataTable.Rows.Count > 0)
            {
                completeExistingImprovementData = OracleMapper.MapDataRowToModel(improvementDataTable.Rows[0]);

                if (!string.IsNullOrEmpty(completeExistingImprovementData.USER_COMMENTS))
                {
                    completeExistingImprovementData.USER_COMMENTS = HebUtils.Ascii7BitToUnicode(completeExistingImprovementData.USER_COMMENTS);
                }
            }

            return completeExistingImprovementData;
        }

REST Service reader using HttpWebRequest and HttpWebResponse (now deprecated, prefer using HttpClient)

Instructions: The key component here is Newtonsoft JSON library, specifically the SelectToken method. Use it to specify the path from where you want to start extracting your data from the json object the service serves.

Usage:
public class RESTServiceBL  
{  
     public Response<List<TServiceModel>> CallRESTService<TServiceModel>(RESTServiceRequest request)  
     {  
       return Integrator.GetData<TServiceModel>(request, Enums.DataRequestTypes.RESTService);  
     }  
}  
 using Fnx.Ambulatory.Data.Sql.Actions;  
 using Fnx.Ambulatory.Data.Sql.Contexts;  
 using Fnx.Ambulatory.Entities.Domain.DataEntities;  
 using Fnx.Ambulatory.Entities.Transfer.Request;  
 using Fnx.Ambulatory.Entities.Transfer.Response;  
 using Fnx.Ambulatory.Infrastructure.Common;  
 using Fnx.Ambulatory.Infrastructure.Common.Types;  
 using Newtonsoft;  
 using Newtonsoft.Json;  
 using Newtonsoft.Json.Linq;  
 using Phoenix.CommonUtils;  
 using System;  
 using System.Collections;  
 using System.Collections.Generic;  
 using System.Collections.ObjectModel;  
 using System.Data;  
 using System.Data.Entity;  
 using System.IO;  
 using System.Linq;  
 using System.Net;  
 using System.Reflection;  
 using System.Text;  
 namespace Fnx.Ambulatory.Data.Integrator  
 {  
   public static class Integrator  
   {  
     public static Response<List<T>> GetData<T>(object parameter, Enums.DataRequestTypes requestType)  
     {  
       Response<List<T>> response = new Response<List<T>>();  
       switch (requestType)  
       {  
         case Enums.DataRequestTypes.Database:  
           Enums.DatabaseDataRequestTypes databaseRequestType;  
           AmbulatoryRequestActions ambulatoryRequestAction = new AmbulatoryRequestActions();  
           Enum.TryParse<Enums.DatabaseDataRequestTypes>(parameter.ToString(), out databaseRequestType);  
           switch (databaseRequestType)  
           {  
             //TODO - all the implementation for obtaining data from the DB...  
             case Enums.DatabaseDataRequestTypes.GetTable:  
               AmbulatoryBaseContext db = new AmbulatoryBaseContext();  
               Type type = typeof(AmbulatoryBaseContext);  
               List<PropertyInfo> properties = type.GetProperties(BindingFlags.Public).ToList();  
               Type t = properties.Find(property => property.PropertyType.Name == typeof(T).Name).PropertyType;  
               //    MethodInfo method = t.GetMethod("GenericMethod");  
               //    MethodInfo generic = method.MakeGenericMethod(myType);  
               //    generic.Invoke(this, null);  
               //    data = db.AmbulatoryRequests.ToList<type>();  
               //    //data = //dbContext as IEnumerable<AmbulatoryRequest>;  
               break;  
           }  
           //response.Data = data as List<AmbulatoryRequest>;  
           break;  
         case Enums.DataRequestTypes.RESTService:  
           response = CallRESTService<T>(parameter as RESTServiceRequest);  
           break;  
         case Enums.DataRequestTypes.SOAPService:  
           break;  
       }  
       return response;  
     }  
     private static Response<List<TServiceModel>> CallRESTService<TServiceModel>(RESTServiceRequest request)  
     {  
       Response<object> response = new Response<object> { Data = string.Empty };  
       List<TServiceModel> modelList = new List<TServiceModel>();  
       if (!string.IsNullOrEmpty(request.URI))  
       {  
         Logger.Info(string.Format("About to call REST Service... Service URI: [{0}].", request.URI));  
         response = GetRESTServiceResponse(request.URI);  
         if (response.Status == Enums.ResponseStatus.Success)  
         {  
           try  
           {  
             JArray entries = JToken.Parse(response.Data.ToString()).SelectToken(request.CollectionPath) as JArray;  
             string modelJsonStr = string.Empty;  
             //if no Entry element was found, skip...  
             if (entries != null)  
             {  
               foreach (JToken entry in entries)  
               {  
                 modelList.Add(JToken.Parse(CustomizeEntryString(entry, request.ViewModelPath, request.ReplaceStrings)).ToObject<TServiceModel>());  
               }  
             }  
             // server returned a single Entry JSON  
             else  
             {  
               modelList.Add(JToken.Parse(CustomizeEntryString(JToken.Parse(response.Data.ToString()).SelectToken(request.CollectionPath), request.ViewModelPath, request.ReplaceStrings)).ToObject<TServiceModel>());  
             }  
           }  
           catch (JsonSerializationException ex)  
           {  
             string msg = "Error in parsing REST json response.";  
             Logger.Error(string.Format("{0}\n{1}. Response: [{2}]", msg, ex.Message, response.Data.ToString()));  
             response.Status = Enums.ResponseStatus.Error;  
             response.Message = msg;  
           }  
         }  
       }  
       return new Response<List<TServiceModel>> { ServiceURI = response.ServiceURI, Data = modelList, Message = response.Message, Status = response.Status, MetaData = response.MetaData };  
     }  
     private static Response<object> GetRESTServiceResponse(string requestURI)  
     {  
       HttpWebRequest httpWebRequest = null;  
       ESBServiceResponseMetadata responseMetadata = new ESBServiceResponseMetadata();  
       HttpWebResponse httpWebResponse = null;  
       Stream responseStream = null;  
       StreamReader responseStreamReader = null;  
       StringBuilder httpActivityLog = new StringBuilder();  
       Response<object> response = new Response<object> { ServiceURI = requestURI, Data = string.Empty, Message = string.Empty };  
       bool tryAgain;  
       try  
       {  
         httpWebRequest = WebRequest.Create(requestURI) as HttpWebRequest;  
         Logger.Info(string.Format("HttpWebRequest object created for request [{0}].", requestURI));  
         LogHTTPHeaderData(httpWebRequest.Headers, httpActivityLog);  
         do  
         {  
           tryAgain = false;  
           try  
           {  
             httpWebResponse = httpWebRequest.GetResponse() as HttpWebResponse;  
             LogHTTPHeaderData(httpWebResponse.Headers, httpActivityLog);  
             Logger.Info(httpActivityLog.ToString());  
             try  
             {  
               responseStream = httpWebResponse.GetResponseStream();  
               responseStreamReader = new StreamReader(responseStream);  
               response.Data = responseStreamReader.ReadToEnd();  
               JToken esbServiceToken = JToken.Parse(response.Data.ToString()).SelectToken("feed.ESBServiceResponseMetadata");  
               if (esbServiceToken != null)  
               {  
                 try  
                 {  
                   responseMetadata = esbServiceToken.ToObject<ESBServiceResponseMetadata>();  
                   response.MetaData = responseMetadata;  
                   if (responseMetadata.ResponseStatus == Enums.ESBResponseStatuses.TechnicalError)  
                   {  
                     if (responseMetadata.ResponseDescription.IndexOf("Failed to establish a backside connection") > -1)  
                     {  
                       tryAgain = true;  
                     }  
                   }  
                 }  
                 catch (JsonSerializationException ex)  
                 {  
                   string msg = "Error in parsing REST json response.";  
                   Logger.Error(string.Format("{0}\n{1}. Response: [{2}]", msg, ex.Message, response.Data.ToString()));  
                   response.Status = Enums.ResponseStatus.Error;  
                   response.Message = msg;  
                   Logger.Error(ex);  
                 }  
                 response.Status = responseMetadata.ResponseStatus == Enums.ESBResponseStatuses.Success ? Enums.ResponseStatus.Success : Enums.ResponseStatus.Error;  
                 //response.Message = Enums.GetEnumDescription(responseMetadata.ResponseDescription);  
                 response.Message = responseMetadata.ResponseDescription;  
               }  
               else  
               {  
                 response.Status = Enums.ResponseStatus.Error;  
                 response.Message = "Could not find path 'feed.ESBServiceResponseMetadata' in JSON string.";  
               }  
             }  
             catch (WebException ex)  
             {  
               if (ex.Message.ToLower().IndexOf("timed out") > -1)  
               {  
                 tryAgain = true;  
                 response.FailedAttempts++;  
                 Logger.Error(string.Format("Response timed out. Commencing retry number {0}.", response.FailedAttempts));  
                 Logger.Error(ex);  
               }  
             }  
           }  
           catch (WebException ex)  
           {  
             if (ex.Message.ToLower().IndexOf("timed out") > -1)  
             {  
               tryAgain = true;  
               response.FailedAttempts++;  
               Logger.Error(string.Format("Response timed out. Commencing retry number {0}.", response.FailedAttempts));  
               Logger.Error(ex);  
             }  
           }  
           catch (Exception ex)  
           {  
             Logger.Error(string.Format("Error in getting request's Response Stream object for [{0}].\n{1}\n{2}", requestURI, ex.Message, ex.InnerException != null ? ex.InnerException.Message : string.Empty));  
             Logger.Error(ex);  
             Logger.Info(httpActivityLog.ToString());  
             response.Status = Enums.ResponseStatus.Error;  
             response.Message = string.Format("Error in getting request's Response Stream object for [{0}].\n{1}\n{2}", requestURI, ex.Message, ex.InnerException != null ? ex.InnerException.Message : string.Empty);  
             response.Data = httpActivityLog.ToString();  
           }  
         }  
         while (tryAgain);  
       }  
       catch (Exception ex)  
       {  
         httpWebRequest.Abort();  
         Logger.Error(string.Format("Error in creating HttpWebRequest object for [{0}].", requestURI));  
         Logger.Error(ex);  
         Logger.Info(httpActivityLog.ToString());  
         response.Status = Enums.ResponseStatus.Error;  
         response.Message = string.Format("Error in creating HttpWebRequest object for [{0}].\n{1}", requestURI, ex.Message);  
         response.Data = httpActivityLog.ToString();  
       }  
       finally  
       {  
         if (httpWebResponse != null)  
           httpWebResponse.Close();  
         if (responseStream != null)  
           responseStream.Dispose();  
         if (responseStreamReader != null)  
           responseStreamReader.Dispose();  
       }  
       return response;  
     }  
     private static string CustomizeEntryString(JToken entry, string viewModelPath, List<string> replaceStrings)  
     {  
       string modelJsonStr = string.Empty;  
       modelJsonStr = entry.SelectToken(viewModelPath).ToString();  
       #region Remove unwanted characters  
       /*  
            * In case there are characters that need to be removed from the Json structure the service returns,  
            * so the names of the fields in the Json object match our model field names - and the cast/mapping works  
            */  
       foreach (string replaceStr in replaceStrings)  
       {  
         modelJsonStr = modelJsonStr.Replace(replaceStr, "");  
       }  
       #endregion Remove unwanted characters  
       return modelJsonStr;  
     }  
     private static void LogHTTPHeaderData(WebHeaderCollection webHeaderCollection, StringBuilder httpActivityLog)  
     {  
       for (int i = 0; i < webHeaderCollection.Count; ++i)  
         httpActivityLog.AppendLine(string.Format("\nHTTP header parameter:{0}, Value :{1}", webHeaderCollection.Keys[i], webHeaderCollection[i]));  
     }  
   }  
 }