Vlad's blog

In programming veritas

Web API data shaping

leave a comment »

Data shaping is a simple technic for minimizing traffic travelling from Client of Web API to Server and vice versa. Client can send many requests to the Server for data retrieval and update. Depending on a scenario Client might not be interested in receiving all fields that comprise data object. For instance Client might want to receive only id and description and ignore other ten fields. When we deal with hundreds of objects travelling from Server to Client that can significantly affect application performance.

Optimizing data retrieval (HTTP GET)

Imagine we are developing simple application for managing tasks. Below is task class definition.

public class Task
{
 public int Id { get; set; }
 public string Summary { get; set; }
 public string Description { get; set; }
 public DateTime LastModified { get; set; }
 public DateTime Created { get; set; }
 public string UserCreated { get; set; }
 public string Assignee { get; set; }
 public TaskState State { get; set; }

 public enum TaskState
 {
   NotStarted,
   InProgress,
   Closed
 }
}

API should expose a method for retrieving task by id. So Url should look like this:

http://taskmanager.com/tasks/1

When this request is processed the Server returns the following result.

{
 "id": 1,
 "summary": "Test task summary",
 "description": "Test task description",
 "lastModified": "2015-11-04T08:29:17.5464689-05:00",
 "created": "2015-11-04T08:29:17.5464689-05:00",
 "userCreated": "UserCreated",
 "assignee": "Assignee",
 "state": 1
}

Consider scenario when Client wishes to receive only id plus summary. Ideally Url for achieving this goal should look like below.

http://taskmanager.com/api/tasks/1?fields=id,summary

Server side code is shown below. Optional argument fields is used to specify comma separated list of fields.

[RoutePrefix("api")]
 public class TaskController : ApiController
 {
   private ITaskRepository taskRepository = new TaskRepository();

   [Route("tasks/{taskId}")]
   public IHttpActionResult Get(int taskId, string fields = null)
   {
     try
     {
       var fieldsList = new List<string>();

       if (fields != null)
       {
         fieldsList = fields.ToLower().Split(',').ToList();
       }

       Task task = taskRepository.Find(taskId);

       if (task == null)
       {
         return NotFound();
       }

       object result = GetShapedObject(task, fieldsList);

       return Ok(result);
     }
     catch (Exception)
     {
       return InternalServerError();
     }
   }

Actual filtering is implemented in GetShapedObject method which uses a bit of reflection to create output data object that contains only fields requested by Client.

private static object GetShapedObject(Task task, List<string> fieldsList)
{
   if (fieldsList.Count == 0)
   {
     return task;
   }

   ExpandoObject result = new ExpandoObject();

   foreach (var field in fieldsList)
   {
     var fieldValue = task.GetType()
     .GetProperty(field, 
        BindingFlags.IgnoreCase | 
        BindingFlags.Public | 
        BindingFlags.Instance)
     .GetValue(task, null);

     ((IDictionary<String, Object>)result).Add(field, fieldValue);
   }

   return result;
}

So the following Url will return only id and summary.

http://taskmanager.com/api/tasks/1?fields=id,summary

{
 "id": 1,
 "summary": "Test task summary"
}

Optimizing data update (HTTP PATCH)

In HTTP world PUT is used for updating a resource. By definition Client should specify all fields of data object, otherwise they will be reset to defaults. I.e. if you want to change only summary you should send entire Task object in a body of PUT request.

{
 "id": 1,
 "summary": "New task summary",
 "description": "Test task description",
 "lastModified": "2015-11-04T08:29:17.5464689-05:00",
 "created": "2015-11-04T08:29:17.5464689-05:00",
 "userCreated": "UserCreated",
 "assignee": "Assignee",
 "state": 1
}

Sometimes this is not desirable due to performance considerations or because Client literally does not have all fields available. In order to update only subset of fields you should use HTTP PATCH. PATCH stands for partial resource modification. The standard does not specify exactly how changes must be represented in the request body. If you don’t want to invent a wheel you can use JsonPatchDocument that describes sequences of operations to apply to a JSON document. For instance, if we want to change task summary we need to send the following request.

PATCH http://taskmanager.com/api/tasks/1 HTTP/1.1
Content-Type: application/json-patch+json

[
  {"op":"replace","path":"/summary","value":"New task summary"}
]

In order to add a support of JsonPathDocument in ASP.Net Web API project you can use library Marvin.JsonPatch which is available on NuGet. Server side implementation is below.

[Route("tasks/{taskId}")]
[HttpPatch]
public IHttpActionResult Patch(int taskId, 
[FromBody]JsonPatchDocument<Task> taskPatchDocument)
{
   try
   {
     if (taskPatchDocument == null)
     {
       return BadRequest();
     }

     var task = taskRepository.Find(taskId);
 
     if (task == null)
     {
       return NotFound();
     }

     taskPatchDocument.ApplyTo(task);

     taskRepository.Update(task);

     return Ok(task);
   }
   catch (Exception)
   {
     return InternalServerError();
   }
}

The following line is required in WebApiConfig.Register()

public static void Register(HttpConfiguration config)
{
   config.Formatters.JsonFormatter.SupportedMediaTypes.Add(
     new MediaTypeHeaderValue("application/json-patch+json"));
}

So after we sent PATCH request that changes summary field the response looks like below.

{
 "summary": "New task summary",
 "id": 1,
 "description": "Test task description",
 "lastModified": "2015-11-04T23:42:57.6376424-05:00",
 "created": "2015-11-04T23:42:57.6376424-05:00",
 "userCreated": "UserCreated",
 "assignee": "Assignee",
 "state": 1
}

 

Written by vsukhachev

November 5, 2015 at 5:10 am

Posted in Development

Tagged with ,

async/await in C#

leave a comment »

Concurrency is mandatory part of almost every modern application. In .Net TPL provides solid foundation for organizing different scenarios of asynchronous communication. Starting from MSVS 2012 we have two new keywords to deal with concurrent methods: async and await.

As example consider client server remote communication. The Server maintains a list of jobs that Client must execute. Once client is successfully authorized it starts retrieving jobs one by one. When a job is finished the Client asks the Server to remove this job from the list and retrieves the next job. All communication between Server and Client is asynchronous because the nature of communication protocol. But the logic is definitely sequential:

  • Authorize
  • Retrieve first job
  • Execute first job
  • Finalize first job on the Server
  • Retrieve the next job

Server
We want to update the UI in non blocking fashion. So if authorization takes 3 seconds the UI should be responsible. Below is the implementation of sample Authentication service. It just adds a delay to simulate server side processing.

public class AuthenticationService
{
   public Task<bool> Authenticate()
   {
     return Task.Delay(TimeSpan.FromSeconds(3)).ContinueWith(t => true);
   }
 }

Read the rest of this entry »

Written by vsukhachev

November 24, 2014 at 1:21 pm

Posted in Development

Tagged with

Task Manager: dependency injection

leave a comment »

This is the fourth post of a series of posts on developing Web applications using ASP.Net MVC. The previous two posts are available here:

The source code is available on CodePlex.

In any application, there is the problem of managing dependencies between components. There is no single solution here. In many cases, there is nothing wrong when a client explicitly creates instances of the required services. In our case we need to look at the dependencies between the application layers. The following diagram shows that the Presentation layer depends on the Domain layer and Data Access layer, and Domain layer depends on the Data Access layer.

Indeed, ProjectController, TaskController and AccountController require a reference to the repository to perform CRUD operations. Some operations require a reference to IProjectDeleteService or IUserDeleteService or IUserActionLogger. Implementations of services also require a reference to the repository. Technically, all three layers are represented by three different assemblies. All interfaces are declared as public, while the implementation declared as internal.
One possible solution here is to use a DI container. There are a sufficient number of libraries that provide functionality of DI container, so the best choice is often determined by personal preference. Personally I like Autofac. The idea of ​​modules is useful for isolation of layers. Thus, each layer will be represented by own module, which registers the dependencies. For example, DomainLayer registers services that are provided by Domain layer.

public class DomainLayer : Module
{
    protected override void Load(ContainerBuilder builder)
    {
        builder.RegisterType<UserActionLogger>().As<IUserActionLogger>().InstancePerLifetimeScope();
        builder.RegisterType<ProjectDeleteService>().As<IProjectDeleteService>().InstancePerLifetimeScope();
        builder.RegisterType<TaskUpdateService>().As<ITaskUpdateService>().InstancePerLifetimeScope();
    }
}

Instances of modules are created to initialize DI container at the application startup.

protected void Application_Start()
{
    var builder = new ContainerBuilder();

    builder.RegisterModule(new DataAccessLayer());
    builder.RegisterModule(new DomainLayer());
    builder.RegisterModule(new PresentationLayer());

    IContainer container = builder.Build();

    // Setup DI as default MVC controller factory
    DependencyResolver.SetResolver(new AutofacDependencyResolver(container));
}

Another task that can be solved by DI container is to manage the lifetime of objects. For example, in order to create instance of ProjectController you need the following services.

  • ProjectRepository
  • UserActionLogger
  • TaskManagerDbContext
  • ProjectDeleteService

ProjectRepository also depends on TaskManagerDbContext. We need to make sure we have only one instance of TaskManagerDbContext in current HTTP request. Otherwise every service will get own copy of TaskManagerDbContext.

When a request is made for a service, Autofac can return a single instance within some kind of context, e.g. a thread or an HTTP request. To do this, call InstancePerLifetimeScope() method when registering the dependency.

public class DataAccessLayer : Module
{
    protected override void Load(ContainerBuilder builder)
    {
        Database.SetInitializer(new DatabaseInitializer()); // Initialize database
           
        builder.RegisterType<TaskRepository>().As<ITaskRepository>().InstancePerLifetimeScope();
        builder.RegisterType<ProjectRepository>().As<IProjectRepository>().InstancePerLifetimeScope();
        builder.RegisterType<UserRepository>().As<IUserRepository>().InstancePerLifetimeScope();
        builder.RegisterType<ProjectLogWriter>().As<IProjectLogWriter>().InstancePerLifetimeScope();
        builder.RegisterType<TaskLogWriter>().As<ITaskLogWriter>().InstancePerLifetimeScope();

        builder.RegisterType<TaskManagerDbContext>()
            .As<IUnitOfWork>()
            .As<TaskManagerDbContext>()
            .InstancePerLifetimeScope();
    }
}

Autofac is smart enough to undestand Lazy class which is usually used for lazy initialization. For example, IProjectDeleteService needed to create ProjectController, but in reality it is only required when processing a request to delete a project. In this case it is convenient to use lazy initialization.

public ProjectController(IProjectRepository projectRepository, 
Lazy<IProjectDeleteService> projectDeleteService, 
IUnitOfWork unitOfWork, 
IUserActionLogger userActionLogger, TabContainer tabContainer)
{
    Contract.Requires(projectRepository != null);
    Contract.Requires(unitOfWork != null);

    _projectRepository = projectRepository;
    _tabContainer = tabContainer;
    _projectDeleteService = projectDeleteService;
    _unitOfWork = unitOfWork;
    _userActionLogger = userActionLogger;

    if(log.IsDebugEnabled) log.Debug("Created ProjectController");
}

Generally, Autofac supports a well defined system of relationship types. More information about the features can be found in blog of one of the developers.

Written by vsukhachev

January 8, 2012 at 1:34 pm

Posted in Development

Tagged with ,

Task Manager: data access layer

leave a comment »

This is the third post of a series of posts on developing Web applications using ASP.Net MVC. The previous two posts are available here:

The source code is available on CodePlex.

The data access layer allows application components to abstract from implementation details of a specific database access technology. Since we use ORM, we already have a substantial level of abstraction that is implemented in the ORM, in our case, Entity Framework. In the case of a simple application like Task Manager, perhaps, there is no need for a separate level of abstraction to isolate parts of the EF from other system components. The only advantage here is that we can easily migrate to another ORM, such as NHibernate. On the other hand, it does not require much effort and demonstrates the good approach in terms of application architecture.
In this case, the standard solution is to use a pattern Repository in conjunction with Unit of work .
From a design standpoint it is important to decide which entity requires a separate repository, and what is not. Eric Evans in his book about DDD recommends creating a repository for each aggregate root. In our case we have three aggregate roots: Project, Task, User, respectively, we have three repositories:

  • IProjectRepository
  • ITaskRepository
  • IUserRepository

Since all the repositories have a standard set of CRUD operations, plus methods for searching and filtering, we can define the common interface they must implement.

public interface IRepository<TEntity, TKey>
    where TEntity: IEntity<TKey>
{
    IQueryable<TEntity> All { get; }
    IQueryable<TEntity> AllIncluding(params Expression<Func<TEntity, object>>[] includeProperties);
    TEntity Find(TKey id);
    void Insert(TEntity entity);
    void Update(TEntity entity);
    void Delete(TKey id);
}

All methods should be obvious, except, perhaps, AllIncluding, but I will tell about it later.
Generic interface IRepository has two parameters: TEntity – that defines the serialized class and TKey – type of the primary key (usually int). TEntity must implement the interface IEntity, which defines the Id property to store the primary key values.

public interface IEntity<T>
{
    T Id { get; set; }
}

Pattern Unit of work is implemented using the interface IUnitOfWork, which is available to clients along with a specific repository.

public interface IUnitOfWork
{
    void Save();
}

Read the rest of this entry »

Written by vsukhachev

January 3, 2012 at 7:01 pm

Posted in Development

Tagged with ,

Task Manager: application architecture and domain model

with 5 comments

This is the second post of a series of posts on developing Web applications using ASP.Net MVC. The previous post is available here. The source code is available on CodePlex.

Application architecture

The system has the following layers:

  • Presentation layer
  • Domain layer
  • Data Access layer
  • ORM
  • Database layer

The last two layers (ORM and Database) are more likely to complete the picture, because they have already been implemented by the Entity Framework.


Read the rest of this entry »

Written by vsukhachev

December 29, 2011 at 11:17 am

Posted in Development, Uncategorized

Tagged with ,

Task Manager

leave a comment »

The best way to learn a new framework is to build something with it. I am going to write a series of posts on how to develop Web applications using ASP.Net MVC, and a number of related technologies and libraries:

  • Entity Framework Code First
  • Autofac
  • Automapper
  • Rhino Mocks

The source code is available on CodePlex. The following posts are available at the moment.

In addition to questions related to the use of mentioned technologies will be discussed principles of DDD in designing application architecture, as well as writing unit tests for components of the system.
Task Manager is a small, but a complete application for managing tasks and projects. The user can create a new project or add tasks in the current project.

In order to create new task you must select project from the list of available projects and choose an assignee.

Each user can view a list of your active tasks, as well as a list of all tasks and a list of completed tasks.

Every task has a log.

Users can add their comments to the task.

Written by vsukhachev

December 28, 2011 at 8:18 am

Posted in Development

Tagged with ,

Ext JS – modern framework for creating RIA

leave a comment »

Modern web application has long ceased to be a set of HTML pages, when interaction is reduced to a simple processing on the server. The growing popularity of RIA requires an infrastructure that allows to program complex logic on the client. In connection with this trend towards the creation of JavaScript frameworks that provide a framework for application design in accordance with such well-known patterns like MVC and MVVM.
Ext JS is one of the popular frameworks for building Web applications that look and behave like traditional desktop application. Typically, such an application is a single HTML page container that contains a hierarchy of widgets. A server role is reduced to the processing of AJAX requests and implement the business logic that can be realized only on the server.
To illustrate the above, consider an example using Ext JS. Suppose we are developing a project management application. The basic application abstraction is a task that has the following attributes.

  • Project which is a container of task
  • Description
  • Status (Not Started,In Progress,Closed)
  • Estimation (man hours)
  • Progress (percentage of completion)

Read the rest of this entry »

Written by vsukhachev

December 16, 2011 at 5:14 am

Posted in Development

Tagged with