Friday, July 10, 2020

Purse dynamic JSON with C#

First, let me explain what is we are trying to do in this post. Recently have a problem needed to parse a JSON to a C# object. Few of the fields are fixed whereas few are dynamic in nature. Like in the following JSON-
 
{
  "text": "TestallDay",
  "description": null,
  "recurrenceRule": null,
  "id": 238,
  "recurrenceException": null,
  "allDay": true,
  "startDate": "2020-07-07T05:00:00Z",
  "endDate": "2020-07-08T05:00:00Z",
  "resourceTypeId_52": [
    134
  ],
  "resourceTypeId_49": [
    118,
    124
  ]
}
There can be multiple such properties with resourceTypeId_* where * is a number and these properties contain an array of integers. Here is how we can solve this in one possible way. Declared a class like below-
 
    public class DataModel
    {
        [JsonProperty("text")]
        public string Text { get; set; }

        [JsonProperty("description")]
        public object Description { get; set; }

        [JsonProperty("recurrenceRule")]
        public object RecurrenceRule { get; set; }
        
        [JsonProperty("Id")]
        public int Id { get; set; }
        
        [JsonProperty("recurrenceException")]
        public object RecurrenceException { get; set; }
        
        [JsonProperty("allDay")]
        public bool AllDay { get; set; }
        
        [JsonProperty("startDate")]
        public DateTime StartDate { get; set; }
        
        [JsonProperty("endDate")]
        public DateTime EndDate { get; set; }
       
        [JsonProperty("ResourceTypeIds")]
        public Dictionary<string, int[]> ResourceTypeIds { get; set; }
    }
Here in the class you can see we have a property of type Dictionary where we need to fill in all such properties. We can use normal JSON conversion (in my case used JsonConvert from Newtonsoft) for other properties and we can use Newtonsoft.Json.Linq.JProperty to loop through all such properties - resourceTypeId_34, resourceTypeId_45 etc and fill in the values in integer dictionary. Here is the code-
 
            string json = "{\"text\":\"TestallDay\",\"description\":null,\"recurrenceRule\":null,\"id\":238,\"recurrenceException\":null,\"allDay\":true,\"startDate\":\"2020-07-07T05:00:00Z\",\"endDate\":\"2020-07-08T05:00:00Z\",\"resourceTypeId_52\":[134],\"resourceTypeId_49\":[118,124]}";
            
            var data = JsonConvert.DeserializeObject<DataModel>(json);
            data.ResourceTypeIds = new Dictionary<string, int[]>();
            JObject item = JObject.Parse(json);

            IEnumerable<JProperty> props = item.Properties().Where(p => p.Name.Contains("resourceTypeId"));
            foreach (var prop in props)
            {
                List<int> arrayItems = new List<int>();
                foreach (var arrItem in prop.Value)
                {
                    arrayItems.Add((int)arrItem);
                }
                data.ResourceTypeIds.Add(prop.Name, arrayItems.ToArray());
            }

The output looks like this screenshot-

 
IEnumerable<JProperty> props = item.Properties().Where(p => p.Name.Contains("resourceTypeId"));
            foreach (var (prop, arrayItems) in from prop in props
                                               let arrayItems = (from arrItem in prop.Value
                                                                 select (int)arrItem).ToList()
                                               select (prop, arrayItems))
            {
                data.ResourceTypeIds.Add(prop.Name, arrayItems.ToArray());
            }
Complete source code can be found here

Saturday, July 4, 2020

jqGrid load data from ASP.NET MVC Core controller

In this post, let's see how to bind jqGrid with data from asp.net core MVC with server-side paging. We are using User details as MVC view model. Here is the ViewModel-

    public class UserViewModel
    {
        public string UserId { get; set; }
        public string UserName { get; set; }
        public string Email { get; set; }
        public string Phone { get; set; }
    }

    public class UserPagingViewModel
    {
        public List<UserViewModel> Users { get; set; }
        public int CurrentPage { get; set; }
        public int ItemCount { get; set; }
        public int TotalPages { get; set; }
    }

Here UserPagingViewModel is used for pagination details for the grid.

Following is the controller and action method-

    public class _2168660_How_to_load_jQgrid_with_Json_data_coming_from_database_in_ASP_NET_COREController : Controller
    {
        public IActionResult Index()
        {
            return View();
        }
        [HttpPost]
        public UserPagingViewModel GetUsers(int page, int rows, string sidx, string sort)
        {
         //todo: use sidx and sort is not used, also read data from database 
            UserPagingViewModel model = new UserPagingViewModel();
            model.ItemCount = 500;
            model.CurrentPage = page;
            model.TotalPages = model.ItemCount / rows;

            var users = new List<UserViewModel<();
            for (int i = 0; i < rows; i++)
            {
                users.Add(GetRandomData.GetRandomDataObject<UserViewModel>());
            }
            model.Users = users;
            return model;
        }
    }

Here, GetRandomDataObject is not included in the source code, what it does is simply creates some rendom data list.

Below is the view part. jQuery reference is missing which you need to include. In my case it is included in the layout page. Here jsonReader is used to convert response to jqGrid desired.

<link crossorigin="anonymous" href="https://cdnjs.cloudflare.com/ajax/libs/jqueryui/1.12.1/jquery-ui.min.css" integrity="sha512-aOG0c6nPNzGk+5zjwyJaoRUgCdOrfSDhmMID2u4+OIslr0GjpLKo7Xm0Ao3xmpM4T8AmIouRkqwj1nrdVsLKEQ==" rel="stylesheet"></link>
<link crossorigin="anonymous" href="https://cdnjs.cloudflare.com/ajax/libs/free-jqgrid/4.15.5/css/ui.jqgrid.min.css" integrity="sha512-xAIWSSbGucVRdutqUD0VLDowcMF/K8W87EbIoa9YUYB4bTyt/zeykyuu9Sjp0TPVdgrgGgBVCBowKv46wY5gDQ==" rel="stylesheet"></link>
<link crossorigin="anonymous" href="https://cdnjs.cloudflare.com/ajax/libs/free-jqgrid/4.15.5/plugins/css/ui.multiselect.min.css" integrity="sha512-UuhJihFIXhnP4QEzaNXfLmzY9W3xoeTDATm0buV4wb2qJKoikNn568f0zA5QmrX0sp6VZzqE6fffvsTYU34tGA==" rel="stylesheet"></link>

<script crossorigin="anonymous" integrity="sha512-uto9mlQzrs59VwILcLiRYeLKPPbS/bT71da/OEBYEwcdNUk8jYIy+D176RYoop1Da+f9mvkYrmj5MCLZWEtQuA==" src="https://cdnjs.cloudflare.com/ajax/libs/jqueryui/1.12.1/jquery-ui.min.js"></script>
<script crossorigin="anonymous" integrity="sha512-xt9pysburfYgFvYEtlhPDo8qBEPsupsDvWB8+iwspD+oQTvAdDEpA1aIKcH6zuABU528YitH6jtP0cAe5GrwKA==" src="https://cdnjs.cloudflare.com/ajax/libs/free-jqgrid/4.15.5/jquery.jqgrid.min.js"></script>

<table id="dataTable"></table>
<div id="pager"></div>

<script>
    jQuery("#dataTable").jqGrid({
   	url:'@Url.Action("GetUsers", "_2168660_How_to_load_jQgrid_with_Json_data_coming_from_database_in_ASP_NET_CORE")',
	datatype: "json",
    mtype: 'POST',
   	colNames:['User id','User name', 'Email', 'Phone'],
   	colModel:[
   		{name:'userId',index:'userId', width:100},
   		{name:'userName',index:'userName', width:200},
   		{name:'email',index:'email', width:80, align:"right"},
   		{name:'phone',index:'phone', width:80, align:"right"}		
   	],
   	rowList:[10,20,30],
   	pager: '#pager',
    width: '700',
    rowNum:10,
   	sortname: 'userId',
    viewrecords: true,
    sortorder: "desc",
    jsonReader: {
        repeatitems: false,
        root: function(obj) { return obj.users; },
        page: function(obj) { return obj.currentPage; },
        total: function(obj) { return obj.totalPages; },
        records: function(obj) { return obj.itemCount; },
        id: "userId"
    },
    caption:"JSON Example"
});
jQuery("#list2").jqGrid('navGrid','#pager',{edit:false,add:false,del:false});
</script>

You can clone the working copy from Github

Thursday, September 27, 2018

Fortify scan with sourceanalyzer using Jenkins - Unable to load build session with ID "sample_id"

Are you running fortify scan through sourceanalyzer for MSBuild? Are you running it with Jenkins? Jenkins node is running with Windows? May be following a process describe here.

Following steps working fine if you are running with powershell or cmd, but not working when you run with Jenkins? Scan is failing on scan step?
sourceanalyzer -b fortify_sample -clean
sourceanalyzer -b fortify_sample msbuild Fortify.Samples.sln /t:ReBuild
sourceanalyzer -b fortify_sample -scan -f result.fpr
Saying-

[error]: Unable to load build session with ID " fortify_sample". See log file for more details.

Provable solution:
Please check the output of the MSBuild step. There will be some .txt file generated for the build step. Please check the location of the files generated. If the files are not generated, that means there are some permission issue of the user under which Jenkins service or agent service is running.

In my case the issue was my Jenkins service was running under local system and it was trying to write to C:\Windows\system32\config\systemprofile\AppData.

I changed service running user id to a service account and started working fine.

How to run fortify scan for dotnet solution using MSBuild

It's simply a 4 stage process.

Step 1: Clear previous scan build
sourceanalyzer -b build_id  -clean
Here build id is an unique string that represent identification of a particular scan in the system (in our case the system is fortify-jenkins-node) where it is run. Ideally this is unique to a solution file e.g.
sourceanalyzer -b appointment_api -clean
Step 2: Analize solution
sourceanalyzer -b build_id msbuild mysolution.sln /t:ReBuild
e.g.
sourceanalyzer -b fortify_sample msbuild Fortify.Samples.sln /t:ReBuild
Step 3: Generate report
sourceanalyzer -b build_id -scan -f result.fpr
e.g.
sourceanalyzer -b fortify_sample -scan -f result.fpr
This will run the scan in local system. We can run scan in fortify server, we need to use a different command in that case, which is cloudscan.

Step 4: Upload report
This step upload report (*.fpr) file to fortify server. This step is needed if we are running local scan. If we are running cloud scan then its not needed.
fortifyclient -url http://fortify.nextgen.com/ssc -authtoken "xxxxxx-xxxxxxxxx-xxxxxxxx-xxxx-xxxxxx" uploadFPR -file result.fpr -project "Application_name" -version 9.9
Here
  • URL is URL for fortify server, the system from where this command is run, should have access to fortify server
  • authtoken is a token type "uploadFPR" which we can get it generated from fortify server. If we dont have one, can contact to AppSec for the same
  • Application_name is name of the application that is created in fortify server for current application. Ideally one solution will have one application in fortify server
  • Version is version number of the application in fortify server.
e.g.

fortifyclient -url http://fortify.myserver.com/ssc -authtoken "038bff7e-7e8c-4007-9aXX-748XXXX1a" uploadFPR -file result.fpr -project "fortify_sample" -version 1.0

Tuesday, August 7, 2018

Microsoft.WebApplication.targets not found with MSBuild for Build Tools for Visual Studio 2017

Was working with Jenkins integration and was using MSBuild for Build Tools for Visual Studio 2017(which by default gets installed into C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MSBuild\15.0). While building a web application developed with framework 4.7.1 (in my case). Class library was getting build successfully but was getting following error for web application.
error MSB4226: The imported project "C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MSBuild\Microsoft\VisualStudio\v15.0\WebApplications\Microsoft.WebApplication.targets" was not found. Also, tried to find "WebApplications\Microsoft.WebApplication.targets" in the fallback search path(s) for $(VSToolsPath) - "C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v15.0" . These search paths are defined in "C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MSBuild\15.0\Bin\MSBuild.exe.Config". Confirm that the path in the declaration is correct, and that the file exists on disk in one of the search paths.
Resolution-

In my case I have downloaded Build Tools for Visual Studio 2017 from https://visualstudio.microsoft.com/downloads/#build-tools-for-visual-studio-2017(search inside All downloads). Downloaded exe will have name like vs_buildtools__1674480101.1516182736.exe.
But while installing build tool I have not selected the following option(Web development build tools)-



Steps:
  • Download build tools from above and rename it to vs_buildtools.exe
  • Open command prompt and CD to the same folder where vs_buildtools.exe is
  • Run the following command- vs_buildtools.exe --add Microsoft.VisualStudio.Workload.WebBuildTools
This command will mark Web development build tools preselected in the installation window. Complete the installation and you should get the issue resolved.

Monday, November 21, 2016

Rethinkdb replace multiple fields with object and array as nested field type

I came across a requirement where I need to replace multiple fields in a document. In my case the document has two fields, one as object and other as array. What was needed is modify the nested array and object in the same query.

Let’s take an example document for this case-
{
 "badgeCount" : {
  "09cf79ad-cce7-4826-8a66-e0653eabae4e" : 0,
  "5cdf9a50-b4e3-4240-8ddc-979b25820745" : 1
 },
 "createdDate" : new Date(),
 "deleted" : false,
 "roomName" : "my room",
 "description" : "my room description",
 "id" : "c58f3c08-4d84-41c7-b705-88cd081dfa04",
 "joinedUserIds" : [
  "09cf79ad-cce7-4826-8a66-e0653eabae4e",
  "5cdf9a50-b4e3-4240-8ddc-979b25820745"
 ]
}
This document is about a room, where a user can join a room via joinedUserIds and they have badgeCount which says how many new messages are there in the room. Each item in the array joinedUserIds is a use id and there is a property of the same user id in the badgeCount object.

So, in my case what was needed, when a user leaves a room we need to remove the user id from joinedUserIds and also from badgeCount.

Solution goes like this-
r.db('test').table('room').get("c58f3c08-4d84-41c7-b705-88cd081dfa04").replace(function (s) {
 return s.without({
  badgeCount : {
   "09cf79ad-cce7-4826-8a66-e0653eabae4e" : true
  }
 }).without("joinedUserIds").merge({
  joinedUserIds : s("joinedUserIds").filter(function (id) {
   return id.ne("09cf79ad-cce7-4826-8a66-e0653eabae4e");
  })
 })
})
We have solved this by replace with chained without. First without is removing 09cf79ad-cce7-4826-8a66-e0653eabae4e from badgCount. Result of first without is, it will remove 09cf79ad-cce7-4826-8a66-e0653eabae4e from badgeCount object. Second without removes joinedUserIds and then adds it back with merge and filter.

Tuesday, April 26, 2016

SQL like IN clause in RethinkDB

Let’s consider a simple example of product. We need to get all product where product ids in a given list. A product document goes like this-
[
      {
         “ID”:0,
         “Name”:”Bread”,
         “Description”:”Whole grain bread”,
         “ReleaseDate”:”1992-01-01T00:00:00″,
         “DiscontinuedDate”:null,
         “Rating”:4,
         “Price”:”2.5″
      },
      {
         “ID”:1,
         “Name”:”Milk”,
         “Description”:”Low fat milk”,
         “ReleaseDate”:”1995-10-01T00:00:00″,
         “DiscontinuedDate”:null,
         “Rating”:3,
         “Price”:”3.5″
      },
      {
         “ID”:2,
         “Name”:”Vint soda”,
         “Description”:”Americana Variety – Mix of 6 flavors”,
         “ReleaseDate”:”2000-10-01T00:00:00″,
         “DiscontinuedDate”:null,
         “Rating”:3,
         “Price”:”20.9″
      },
   …
   ]
An equivalent SQL statement in this case will be –
SELECT ID, Name, Description from Products WHERE ID IN (1,2)
To implement this we can use filter with expr and map like below-
r.table('products').filter(function(product) {
        return r.expr([1,2]).contains(product('ID'))
    }).map(function (product) {
        return {
            id : product('ID'),
            name : product('Name'),
            description: product('Description')
        }
    });