Data Caching with .Net 4.0 and Asp.Net MVC – part 1

Posted on August 14, 2010 by Steve | Permalink 7

The purpose of this article is to demonstrate a sort-of real-world example of how to use the new .Net 4 Caching Framework, utilising some good extensible programming practises. This isn’t specifically targeted at MVC programmers, but I am using Asp.Net MVC as the host to show the results of fetching data via the cache. If you want, you can download the demonstrate app and walk through it with me.

First of all, an overview of the structure of the application and what we’re going to be doing. The example application and the code snippets below are written in C#.

  • Creating a bog-standard MVC 2 application
  • Use the Entity Framework to read our data from the database, and utilise a repository pattern to wrap up this database interaction
  • Wrap up the new .Net 4.0 caching objects ObjectCache and MemoryCache to give ourselves another extensibility point, and aid unit testing
  • Tie all this together using the MVC front-end

By the way – the tools I am using here are Visual Studio 2010 Professional (Express would work too, although I haven’t tested it), Asp.Net MVC 2, .Net Framework 4 and Sql Server 2008 Express Edition. Although I’ve mentioned it above, there isn’t any unit testing being done in the downloadable demo application (as it’s not the real focus of this article) but, as you will see, the techniques we will implement lend itself very well to being unit testable and you are free to carry out this testing if you wish.

First of all, let’s set up the database and our entities!

Read more »

Articles , , , , , ,

LinqToSql and Nullable Varchar Fields

Posted on June 20, 2010 by Steve | Permalink 0

I came across an interesting problem this week regarding writing a LINQ query against a LinqToSql data context which contained a Varchar NULL field. The problem was, I had added this field manually into the data context instead of dragging the tables onto the designer from the server explorer. What I had neglected to do was set this field, which was nullable in the database schema, to nullable within the data context. It might be reasonable to think that this might not matter since you can set a string to null in code anyway without doing anything special to the variable (as opposed to specifically marking an int as nullable, for example).

If you don’t do this, the following can happen:

The specific query I was trying to write involved a simple keyword-matching query across a number of string fields, something like the following:

from player in DataContext.Playerswhere
(player.Name + " " + player.Email + " " + player.Nickname)
.ToLower().Contains(keywords)
select player;

This simply concatenates the fields I want to search together and compares them against my keywords. In this case, ‘Nickname’ is optional and therefore nullable in the database. If the query comes across a null entry, it effectively voids the entire row and returns nothing. So the query is changed to something like this:

from player in DataContext.Players
where (player.Name + " " + player.Email + " " + (player.Nickname ?? ""))
.ToLower().Contains(keywords)
select player;

..which means if Nickname is null it concatenates an empty string instead. That is, if you have the field set as nullable within your LinqToSql data context. If you don’t you’ll find that the resulting SQL will look like the following:

SELECT [t0].[Id], [t0].[Name], [t0].[Email], [t0].[Nickname]
FROM [Test] AS [t0]WHERE (((([t0].[Name] + @p0) + [t0].[Email]) + @p1) +
((CASE WHEN 0 = 1 THEN [t0].[Nickname]
ELSE CONVERT(NVarChar(50),@p2)
END))) LIKE @p3

As you can see, for some reason the above SQL could be generated, which still won’t work due to the obvious flaw in the logic (WHEN 0=1). If you have correctly set up your nullable string properties you should see the following being generated, which is more correct:

SELECT [t0].[Id], [t0].[Name], [t0].[Email], [t0].[Nickname]
FROM [Test] AS [t0]WHERE (((([t0].[Name] + @p0) + [t0].[Email]) + @p1) +
((CASE WHEN [t0].[Nickname] IS NOT NULL
THEN CONVERT(NVarChar(50),[t0].[Nickname])
ELSE CONVERT(NVarChar(50),@p2)
END))) LIKE @p3

On reflection, this is exactly the way that it should work. However, it has caught me out enough times to cause me to write about it, so that I don’t forget it again!

Snippets , , ,

Creating self-hosted WCF Data Services

Posted on June 18, 2010 by Steve | Permalink 0

This question came up the other night at the June NEBytes meeting after a presentation by Iain Angus (Black Marble) on WCF Data Services – can you host a WCF Data Service outside of IIS, inside a console application for example?

Since WCF Data Services are based upon the normal WCF architecture, you can self-host them just like any other WCF Service using a DataServiceHost. As a quick example, try this:

1. Create an entity

public class Person
{
 public int ID { get; set; }
 public string Name { get; set; }
}

2. Create a custom data context to return some data

public class ExampleDataContext
{
 public IQueryable People
 {
  get
  {
   return new List()
   {
    new Person() { ID = 1, Name = "Steve"},
    new Person() { ID = 2, Name = "Dave"}
   }.AsQueryable();
  }
 }
}

3. Create a class which inherits from DataService<T> and specify your custom context. Also implement the static InitializeService() method and set up your access rules. This is the stuff that would normall be in your .svc code-behind in the IIS-hosted version:

public class PersonDataService : DataService
{
 public static void InitializeService(DataServiceConfiguration config)
 {
  config.SetEntitySetAccessRule("*", EntitySetRights.AllRead);
 }
}

4. Finally, in your startup code create an instance of a DataServiceHost, specifying the typeof DataService and the Uri to host on. Open the host and wait for a key-press to close it again:

DataServiceHost host = new DataServiceHost(typeof(PersonDataService), new Uri [] { new Uri("http://localhost:999")});

try
{
 host.Open();

 Console.WriteLine("Host is running; press a key to stop");
 Console.ReadKey();

 host.Close();
}
catch (Exception)
{
 host.Abort();
 throw;
}

Using this example, you can now browse to http://localhost:999 in your browser and retrieve your data. You can also create a client proxy against this service at the same Uri just as normal.
You can read more information on hosting WCF Data Services in IIS or otherwise on MSDN.

Download the example code

Articles , , ,

New WCF Article on Codeproject.com

Posted on March 3, 2009 by Steve | Permalink 0

Finally I found I had something to write about; an article on creating a remote file store using WCF, and it’s hosted right now over at CodeProject.com. It basically shows you how to create a service-based remote file repository using the Windows Communication Foundation, which has been part of the .Net Framework since version 3.0.

Enjoy the article!

Articles , , ,

Simple Object Conversion Pattern

Posted on December 18, 2008 by Steve | Permalink 0

Here’s a quick conversion pattern you can use if you’ve got a lot of object conversion going on within your project, especially where databases and web services are involved. Bear in mind that this is essentially what the adapter pattern handles, but doesn’t involve any sort of inheritence. It’s like a cruder version of that pattern. It’s also not as quick to use as a simple linq projection. However, you might want to do that tranposition more than once, so you’d want to stick that logic into a utility method or something similar.This approach just formalises that utility method and enables you to keep all these kinds of conversions in one place; under one roof, if you will.

My solution involves an interface and (optionally) a utility method. It’s handy for when you’ve got a set of data objects that you want to easily transpose and modify in order to send through a web service call to a client. Even if Linq To Sql (for example) does support such serialisation with WCF services without having to do much work, this often isn’t desirable as generally you, as the programmer, want more control over exactly which data is sent across the wire. You might want to do this to guarantee packet sizes, for example, not to mention keeping the interface intact since your domain objects will change.

Here’s the interface for the converter. Objects which implement this interface support converting an object from one type to another:

public interface IConverter&lt;TInput, TOutput&gt;
{
 TOutput Convert(TInput input);
}

Very simple. As an example, an object which converts a series of integers to a series of strings might look like this:

public class IntConverterConverter : IConverter&lt;int, string&gt;
{
 public string Convert(int input)
 {
  return "Number: " + input.ToString();
 }
}

The optional utility method I mentioned earlier is simply a wrapper for converting a whole bunch of inputs in one go. It produces a list of outputs where each item in the list has been converted from each item in the input list. It’s also been implemented as an extension method, so this will work with anything which implements the IEnumerable<T> interface:

public static IEnumerable ConvertAll&lt;TConverter, TInput, TOutput&gt;(this IEnumerable inputs)
 where TConverter : IConverter<TInput, TOutput>, new()
{

 IConverter<TInput, TOutput> converter = Activator.CreateInstance(typeof(TConverter)) as IConverter;
 List<TOutput> outputList = new List<TOutput>();

 foreach (TInput input in inputs)
 {
  outputList.Add(converter.Convert(input));
 }

 return outputList;
}

Here’s an example of using this pattern to convert a bunch of domain objects to a set of data transfer objects (DTOs) with the intention of sending these objects to a WCF client. First, this is what the domain objects look like:

And here’s the DTO I’m going to convert them to:

[DataContract]
public class PersonDto
{
 [DataMember]
 public string Name { get; set; }

 [DataMember]
 public string HomeTown { get; set; }

 [DataMember]
 public string [] EmailAddresses { get; set; }
}

The converter object to convert the domain object to the DTO is:

public class PersonDtoConverter : IConverter&lt;Person, PersonDto&gt;
{
 public PersonDto Convert(Person input)
 {
  return new PersonDto()
  {
   Name = input.Name,
   EmailAddresses = input.EmailAddresses.Select(e =&gt; e.Address).ToArray(),
   HomeTown = input.HomeTown.Name
  };
 }
}

And a quick example of converting a set of Person domain objects to a set of DTOs:

using (ExampleDataContext dc = new ExampleDataContext())
{
 var people = from p in dc.Persons

       select p;

 var convertedPeople = ConversionHelper.ConvertAll(people);

 foreach(var person in convertedPeople)
  Console.WriteLine(person.Name);
}

So now instead of a hierarchy of objects representing a person and all the information we can associate with him, we’ve reduced it to a simple class with a fixed set of properties, allowing the domain object to change freely without affecting any public interfaces and services you might have. If the domain objects do change, you should only have to reflect that change in your conversion code in one place. You could go a step further and create an interface for each DTO which makes this even more flexible. I’ve produced a small project containing all of the above code and the database schema for the DataContext used here, should you wish to borrow or play around with this at your own leisure.Download the example project

Articles ,

LinqToLfsWorld – A look at the demo website

Posted on July 24, 2008 by Steve | Permalink 0

Now that LinqToLfsWorld has been out for a couple of days, I’ve got time to explain a bit more about the features, using the demo website as an example.

The v0.8 Preview release contains the code library and the demo website for you to have a look at the interface first-hand. I should probably mention that for release 1.0 this will probably change; I’ll make the demo website a separate and optional download, and the library in a separate download on its own. This blog post will use that demo website as an example.

The Code
Here’s the code from the code-behind file, which I’m going to be referring to throughout this post:

If you’ve ever used Linq To Sql, you would be familiar with the methodology at work here. Essentially, to query LfsWorld you would do the following:

  1. Create a new instance of the LfsWorldContext
  2. Write a LINQ query to retrieve results using one of LfsWorld’s services (hosts, racer stats, world records, etc).
  3. Step through the returned data or bind to a data control.
  4. Dispose the instance of LfsWorldContext

Let me explain a few things about this, starting with the LfsWorldContext itself.
The context has a strongly-typed collection representing each of the services exposed in v1.4 of the pubstat service. To make these queries, it requires that you supply it your credentials which you use to access LfsWorld (either your username and password, or your ident key if youhave one). You can do this in two ways – through the LfsWorldContext constructor, or through the configuration file (see Configuration below).

The context has CachingEnabled set to true by default – I’ve explicitly set it above just for clarity. Whenever you make a request to LfsWorld, the response gets cached for later. This is done for performance reasons and, given that LfsWorld access isn’t free for premium users, protected your wallet! The cache time can be configured using the CacheTime property on the context – just set this to a Timespan value of your choice.

Another caching strategy at work here is to protect those users which do not have premium access to the LfsWorld service. Any request made within five seconds of another one will throw an exception, and no request is made to LfsWorld. This is also persistant across different context instances. This behaviour can of course be switched off if you do have premium access to the service, by setting the context’s PremiumPubstatAccess property to true, which will allow you to make requests as quickly as you like. This property is false by default.

The next part at work here deals with an event handler, which allows you to detect when any request has been made through the context instance. This allows you to retrieve information about the url which was sent to LfsWorld, the response it got back, whether it was cached or not and the size of the data which was returned.


This piece of code actually builds the query. Note that it does just build the query and does not actually execute it. This is the nature of Linq queries – the data isn’t retrieved until the results are evaluated (by using a for-each loop or converting to a list, for example). You should also note that the queries here can be quite fickle. This is a limitation of the current version of LinqToLfsWorld and will be improved by the 1.0 release.

What I mean by that is it is a bit limiting as to what you could use in that query. For example, if I was querying racer stats I would use a query like the following:

var stats = from s in context.RacerStats
                where s.RacerName == "elkdanger"
                select s;

This would work fine, as would:

string name = "elkdanger";
var stats = from s in context.RacerStats
                where s.RacerName == name
                select s;

But something like the following wouldn’t (imaging GetRacerName() is some method I have created):

var stats = from s in context.RacerStats
                where s.RacerName == GetRacerName()
                select s;

What you would find is the context would try to sent a url to LfsWorld which queries for a racer’s statistics, would would pass through an empty racer name parameter, resulting in LfsWorld returning you no data. In fact, you’ll find that any method operation inside that query would produce a similar result.

Another thing you’ll notice is that for the initial query, only the parameters that LfsWorld accepts for a given action will affect the query (for example, querying racer statistices only accepts a “racer” parameter) . This by design, and the intention is that users of LinqToLfsWorld will make their initial query using only the parameters available to them, evaluate that query and then use plain Linq To Objects to deal with any further filtering that they might need. Thisis the most flexible way and will give the programmer ultimate control over the resultset.

At this point in the example, we evaluate the query by converting it to a list, and selecting only those hosts which are currently populated. At this point in the example we have a plain old list of host objects in memory, which we can treat just like any other object.

Configuration
One thing you can’t see in the code above is the configuration. LinqToLfsWorld allows you to use a custom configuration section to specify the same settings for all of your LfsWorldContext instances (you can override them of course), making it easier to use this libraryfrom multiple places in the same project.
The configuration file for the demo website project look like this:

These settings will apply to any instance of the LfsWorldContext that you create, but of course can be overridden by any instance also. The only required attribute here is authenticationMode, which should be set to ‘IdentKey’ or ‘UsernamePassword’. Hopefully it should then follow that if you set authenticationMode to ‘IdentKey’, you must also supply the idenyKey attribute. Likewise for UsernamePassword authentication mode; you must supply the pubstatUsername and pubstatPassword attributes. For the next release an xml schema file will be available which will give you intellisense on this configuration section, so you don’t have to remember these. For now you can refer to the example site.

The next post will more than likely be an update on how progress is being made towards the release of v1.0, once I have improved the expression parsing side of things. Enjoy LinqToLfsWorld so far!

Articles , , , ,

Querying LfsWorld with LinqToLfsWorld

Posted on June 28, 2008 by Steve | Permalink 0

In this post I’m going to show you all the different types of queries you can perform with LinqToLfsWorld. For an overview of what the library is and how it works, please feel free to read an earlier blog post which takes you through the demo website (included with the library download file).

The Queries

With the library you can perform all the queries that you would normally be able to using a plain url to LfsWorld (according to v1.4 of the pubstat service, which is the latest version at the time of release). These queries are accessed through the LfsWorldContext. All the examples below assume I have already initialised my context with the name lfswContext, and called Dispose() when I’ve finished with it. Disposing the context is important so that it can free up memory and unregister internal event handlers. If you don’t dispose the context, you might find some weird things happening if you’ve handled the context’s RequestMade event.

Racer Stats
Required parameters: RacerName

var query = from stat in lfswContext.RacerStats
where stat.RacerName == "elkdanger"
select stat;

This query will either return no results or one result; LinqToLfsWorld supports the use of the Single and SingleOrDefault methods, so retrieving a single entity instance is quite easy.

Racer Fuel
Required parameters: RacerName

var query = from fuelInfo in lfswContext.RacerFuels
where fuelInfo.RacerName == "elkdanger"
select fuelInfo;

This query returns information about the fuel usage of a racer, on a given track in a given car.

Personal Bests
Required parameters: RacerName

var query = from pb in lfswContext.PersonalBests
where pb.RacerName == "elkdanger"
select pb;

World Records
Required Parameters: None
Optional parameters: Car, TrackIdent

var query = from wr in lfswContext.WorldRecords
where wr.Car == "XRG"
&& wr.TrackIdent == "000"
select wr;

The values for TrackIdent and Car are exactly the same as the pubstat service normally accepts, according to the pubstat documentation on lfsworld.net.

Hotlaps
Required parameters: RacerName

var query = from hl in lfswContext.Hotlaps
where hl.RacerName == "elkdanger"
select hl;

Hotlap Chart
Required parameters: TrackIdent, Car
Optional parameters: SteeringMethod

var query = from hlc in lfswContext.HotlapChart
where hlc.TrackIdent == "000"
&& hlc.Car == "XRG"
&& hlc.SteeringMethod == "ms"
select hlc;

The values for TrackIdent, Car and SteeringMethod are exactly the same as the pubstat service normally accepts, according to the pubstat documentation on lfsworld.net.

Hotlap Log
Required parameters: None
Optional parameters: Steering, LogFilter, StartTime, Lines

var query = from hll in lfswContext.HotlapLog
where hll.Steering == "ms"
&& hll.LogFilter == HotlapLogFilter.Top10
&& hll.StartTime == DateTime.Now.AddMonths(-1).ToUnixTimestamp();
&& hll.Lines == 90
select hll;

It’s worth mentioning here the call to ToUnixTimestamp(). By importing the LinqToLfsWorld.Extensions namespace. As the name suggests, this method is an extension on the .Net DateTime object which translates it into a unix timestamp value.

Similarly, by calling DateTimeExtensions.FromUnixTimestamp(timestamp) you can convert a timestamp value into a DateTime object; again after importing the LinqToLfsWorld.Extensions namespace.

Hosts
Required parameters: None

var query = from host in lfswContext.Hosts select host;

Teams
Required parameters: None

var query = from team in lfswContext.Teams select team;

That is all the queries that can be performed with the LinqToLfsWorld library. Of course all of these queries can be made using the normal method syntax, such as in:

var query = lfswContext.RacerStats
.Where(stats => stats.RacerName == "elkdanger").Single();

And of course, such is the nature of Linq, all these queries are only ever executed on the server when the results are enumerated.

Articles , , , ,

LinqToLfsWorld v1.0.2 Released!

Posted on June 24, 2008 by Steve | Permalink 0

Just a quick post to say that LinqToLfsWorld 1.0.2 has been released – you can grab it from the project homepage.

This version of the library adds:

  • Support for serialization and deserialization to/from Xml and Json
  • A new demo website page, demonstrating the serialization features.
  • Some helper constants for use in your car type, track ident and steering method queries
  • A couple of bug fixes

The entity classes also support exposure through WCF services – I haven’t added a test for this scenario yet to the project, but have verified in my own quick test project that it does work as expected.
I’ve written a more detailed account of how the serialization works on the 1.0.2 download page.

Articles , , , ,

LinqToLfsWorld v1.0 Released!

Posted on June 14, 2008 by Steve | Permalink 0

Just a quick post to say that LinqToLfsWorld 1.0 has been released; you can grab it from the project homepage on codeplex.com.

Whilst the interface hasn’t changed, under the bonnet the expression parsing code has been vastly improved and you can now do a lot more with it, compared to the 0.8 release from the beginning of the month.
For example, with the preview code you couldn’t write a query such as:

var stat = from stats in context.RacerStats
where stats.RacerName == GetRacerName()
select stats;

.. the point here being that the expression parser couldn’t evaluate local expressions, i.e. any expression which didn’t contain a parameter expression from inside the query itself (in the example above, this local expression would be the call to GetRacerName()). Now though, all expressions which can be evaluated before the query is even looked at are evaluated down to a constant value. This means you can call any method or access any property you like from inside the query.

Some of the common selection methods are now supported also; you can directly call Single, SingleOrDefault, First, FirstOrDefault, Last, LastOrDefault and Count on an LfsWorldContext query. You can also call Select,provided the query only selects either a single entity or a list of entities. In other words, you cannot create an anonymous type from these queries. This limitation is trivial however, since you can just convert your results into a list and use plain Linq To Objects to select only the fields you need. The example website included with the download demonstrates this.

This released also fixed a number of bugs which were flagged up from testing the preview version, mainly to do with caching and the expression engine.

There are a couple of issues I’d like to sort out with regards to serialization, which will come out as some sort of ’1.0.1′ release perhaps. Over the next week or so I’m also going to give you a full run down of each type of query you can perform with this library on this very site.

If anyone is using the library and finds any issues with it, please direct them to the issue tracker on the project home page.

Articles , , , ,

LinqToLfsWorld v0.8 Preview

Posted on June 2, 2008 by Steve | Permalink 0

After a bit of umming and arrrring I managed to make quite a bit of headway with a project I’ve been mulling over for a while – LinqToLfsWorld. It’s a custom LINQ provider which allows the client to communicate with LfsWorld via its Pubstat web service.

What is that exactly? Live For Speed (LFS) is (primarly) an online racing simulator, and LFSWorld is an online base for checking your racing stats, viewing and comparing hotlaps and watching other races in real time via a neat flash application. Pubstat is it’s webservice which you can query to retrieve information about racers, hosts, teams and hotlaps. You send it a url and it will return either JSON, PHP or XML formatted responses.
Since it’s url based, and it has a fairly simple interface, Ithought it would be a good introduction into finding out what it takesto build a custom LINQ provider. My implementation isn’t great nortotally versatile (yet), but it works and I’m pretty pleased with theproject so far.

The reason the release is tagged as 0.8 Preview is because there’s acouple of issues I’d like to sort out before I give it that ‘release1.0′ milestone; the object documentation (currently there isn’t any)and I’d like to work a bit more on the expression visitor, to make thequeries just a bit more friendly.

So, if you’re interested in having a look, please visit the project page on Codeplex and download the latest release and/or the source code and play aroundwith it. Send me any issues if you want via the issue tracker, and feelfree to contact me with any suggestions or queries relating to theproject.

I’m going to publish a couple more articles over the course of thenext couple of weeks as I improve the library, and to show you a couplemore examples on how to use it.

Articles , , , ,