stop trying to show how smart you are

Sitting feeling bright and somewhat giddy with my own self confidence I decided to dust down my copy of “C++ Template Metaprogramming” by David Abrahams and Aleksey Gurtovoy. Thinking I really need to be the master of something and further thinking C++ template programming was that “thing”. I approached the task with some gusto for at least a few days hours minutes, however, as with most advanced study you really have to be willing to dedicate your life to it. It’s never long before the inevitable gloom of reality sets in and you realise that this is more than an afternoons work. Without much control your brain starts telling you that there are much more important things you could be doing. Thankfully for everyone else on your team you stop and decide that it’s just not worth it.

The fact is that writing code that only those who have decided to dedicate a fair chunk of their life to is never going to be the right choice. There is nothing worse than trying to understand code like this. There are times where it pays to be smart, like when you have a better (much) faster algorithm, or doing something saves you hundreds of lines of code. But most of the time you see this kind of code it’s people just trying to prove how smart they are.

Now C++ is not alone in this. Ruby has exactly the same problem – you can get stuff happening as if by magic. I could be wrong but I imagine most people read code from top to bottom, working line by line, they don’t expect code to be auto generated or happen as some elaborate method_missing technique. Sure I can maybe relax my vitriol for those building frameworks where things are being used in some generic unknown context. However the vast majority of applications out there don’t have to deal with these problems. The biggest problem these applications face is that the developers creating them want to architect some elaborate framework to fit a very specific use case – it almost makes me cry. I’m not sure why as a profession we don’t revel in an approach that oozes simplicity. This is certainly what the smartest developers I’ve working with have always managed to do.

is ruby killing your career?

I’m probably at the point with Ruby where I consider it my programming language of choice (I program in both Ruby and C++ in my day job).

Over the last few years I’ve kind of grown to love Ruby but I’m not really one to get passionate over someone else’s choice of programming language – apart from Java, which, I’m sorry, I hate. However, when it comes to employment, there is no doubt in my mind that being competent in a particular programming language can strongly influence A) getting an interview and B) getting the job.

This is why ruby developers, like me, are killing their career. Sure Ruby is cool and Rails is awesome but do a quick check on job boards and see how many people are looking for a ruby developer. Actually, let me save you the time I’ve done some of the work already.

I’m not claiming this to be scientific in anyway what-so-ever but it does warrant some thought. I only searched using the programming language as a keyword, which, I know, may not give the full story but should convince you there is some merit in the point that I’m trying to make. Additionally (and I suppose somewhat importantly) my search area was restricted to Scotland.

First up I carried out a search on s1jobs.com. The table below gives a summary of the results:

Language Number of jobs matching keyword
Ruby 3
Java 18
C# 26
C++ 9
PHP 7

I then tried a cwjobs.co.uk:

Language Number of jobs matching keyword
Ruby 2
Java 35
C# 45
C++ 45
PHP 4

As you can see, the job prospects for Ruby developers here in Scotland are somewhat dire. Sure, people don’t always look for a particular programming language when employing someone (which is a decent policy) but, as I said above, it helps a lot.

I decided to take my crude search a little further as I thought “Hell, there will be waaaaaaay more cool Ruby jobs in London”. Below we have the results, just cwjobs this time:

Language Number of jobs matching keyword
Ruby 57
Java 792
C# 838
C++ 611
PHP 196

That was kind of disappointing! Ruby still doesn’t do that great – even worse when you realise there were over 200 that mentioned Perl and 150 Python. By the looks of it if you want to maximise your chances of getting a job in the UK, and already doing Java or C# in your day job, you’d be better off learning C/C++ in your spare time.

Is all this going to stop me coding in Ruby? Probably not. Is it worth thinking about for a minute? Yes sure. If I was starting my own company and was hoping to get some developers in then I’m likely to be faced with a problem. Yes you can train people up, but that costs time and money. When they leave it may be worse, as the chances of finding replacements at the required skill level will be difficult. Finding a Java/C#/C++ programmer is bound to be far easier.

So is it all bad news for us Ruby developers? Well not if you plan to move to California – yeah yeah I know I’ve went on about it before. I’m not exactly sure of the popular job boards in the US so I went with the only one I knew off the top of my head, careers.stackoverflow.com. The results for the Bay Area are as follows:

Language Number of jobs matching keyword
Ruby 27
Java 33
C# 10
C++ 23
PHP 17

Maybe this was a skewed sample set but impressive all the same. So moral of the story is if you want to be a well paid Ruby hacker make sure you don’t stay in Scotland :-).

back to the future with c plus plus

Around 6 years ago I was steadily programming away in C++ in Unix using a Sun Workstation and I quickly followed this up with some Windows programming using MFC. During this time I also experimented with PHP creating a small “MVC” style framework long before Rails was even a twinkle in it’s daddy’s eye. Now since then I’ve managed to have quite a varied experience with languages such as Java, C#, and most recently Ruby. However, I now find myself running back 6 years and working with good old C++ again.

So why am I back here? Well to be honest, performance. The stuff that I’m currently working on (Kidney Exchange Problem) requires a fair bit of it – it’s NP-hard optimization problem that that we really do need to find an optimal solution for using an exponential-time algorithm. Anyway, maybe more details on this another day.

With C++ being decidedly “yesteryear” with the cool kids, I was pleased to find that it still seems to have a large user base, a moderately active community, and with Google being heavily invested, it would appear that it ain’t going to die as quickly as people may have though. Despite this, I wasn’t holding out too much hope for finding a wealth of “modern” programming tools, and first on my list was a JUnit/NUnit clone.

After some searching and experimentation (which resulted in some moderate surprise) I came across Google Test – good old Google! In my opinion this is the best C++ unit test framework out there. I did try others (CppUnit, CppUnitLite, and another which I now can’t remember) but Google Test was the easiest to use, has everything I need, and seemed to be the most active in terms of development.

Also, now very old hat, is Boost. Although this has been around for quite some time, back when I was first invested in C++, this just didn’t exist (or if it did it was in its infancy). I just love things like shared_ptr and the Boost Graph Library. I’m sure there will be much much more I will find to use in the coming months in Boost.

Finally, for just now, I also have to put out an honerable mention to Google Mock. I’ve not had a chance to use this much at the moment but, like Google Test, it looks as if it will come in pretty handy.

It’s nice to see that C++ is still living and breath, even if it ain’t so cool any more. However, with the community and C++0x it looks as if it may well just have some sort of future.

sticking with what you know

There comes a time in every programmers life when they have to learn new things and step out the box. Yeah it’s difficult, for sure. It’s all too easy to create the latest application in your software empire, using a language you’ve been developing in for the last 10 years. However, the real problem is thinking this is the only choice. When is it time to abandon this certitude?

First, we cover the forced abandonment. This is when you are pushed kicking and screaming into pastures new, whether you like it or not, i.e. the new job. Here, not only is the new language curve ball thrown (viciously), but you also get whole new set of business rules into the bargain. So what do you do? You program the new language like the old one, only translating the syntax in your head. This is not the best way to learn a language though. Why? Well consider those C programmers trying to program imperatively in Java, Java programmers in JavaScript, C++ programmers in Ruby, and so on. When there is a change in paradigm this mapping strategy just doesn’t work – a similar situation exists with languages that contain a more powerful expression set. It also encourages the behaviour where people learning enough to get the job done, without understanding what is really happening, or that there may have been a better way using “unmappable” language’s features. A better approach would be to write something small, and new, that allows you to explore the language’s features. I’m sure most people can think of something they could write. Furthermore, if you can make it useful to other people, or even your new employer, then everyone’s a winner! This is something I touched on before.

For many people though, this is the only time they will ever consider abandoning. This is sad, and a poor characteristic in a programmer. And to be honest, I just don’t understand it. That’s not to say that I don’t accept that people just do programming as a job, then go home and don’t think about it. However, it’s like most things in life, it’s nice to progress?

As a programmer there will also be other signs that the tide is turning, and you don’t have to be too alert to spot these. Previously I wrote “Perl is Dead, Long Live…Perl?” and being a big Perl fan it was sad to see the language apparently dying, so I know what it’s like. Some signs to look out for may be:

  • the language features are not moving on (Java watch your back) – the people who created it no longer care,
  • the community surrounding the language is dwindling – the people who use it no longer care,
  • there is little in the way of choice when selecting libraries/frameworks – the experts have fled,
  • other programmers have never heard of it – there is no buzz,
  • jobs using it are few and far between – businesses have given up on it, the death kneel.

However, this is all not to say that you give up on your language just because it’s no longer cool – popularity is by no means a great indicator that something will suit your needs. It need not be the case that you give up on your language of choice, instead it could be that you contribute and drag the language forward. But be careful with this one.

Finally, any decent employer will want to see that you are continually developing your skill set – their business needs are continually evolving, so why aren’t you? You are much more likely to land a better job if you contribute to your own education in some way. It looks good and it’s also something to talk about.

So go out and learn something new today, and stop sticking with what you know.

charting success

In my continued exploration of C# I recently had the opportunity to investigate the Google Chart API using Google Chart Sharp. Both the C# API and Google Chart itself are great. For those unfamiliar with Google Chart here is a quick summary.

Google Chart allows you to dynamically generate many of the popular types of chart – currently the API includes support for Line Charts, Bar Charts, Pie Charts, Venn Diagrams, plus many more. Check out the full list of chart types here.

Charts are generated via a GET request using a simple URL, where various GET parameters can be supplied to customize the chart. For example, to construct a pie chart that describes web browser usage for a particular page in which FireFox has 60%, and IE 40% of the total views, we use the following URL:

http://chart.apis.google.com/chart?cht=p3&chd=t:60,40&chs=250x100&chl=FireFox|IE

Typing this URL into your browser will show the chart below:

Pie Chart with two data points

Pie Chart with two data points

I’m sure many of you will recognise the chart design from Google Analytics.

Not only is the Chart API free to use (but not to abuse!), it also generates pretty nifty looking charts.

The charts can also be customised in many different ways including size, colour, axis, etc. Check out the documentation for this on the Google Chart API page linked to above. To show how simple it is to change the colours, take a look at the chart below:

Pie Chart with custom colours

Pie Chart with custom colours

Here the only change required to add custom colours is an additional HTML GET parameter – I simply added chco=9913CE,D3A4E5 to the end of the URL given above.

Now on to the C# side of things.

As you can probably guess, the C# wrapper for Google Chart simply generates a URL. For example, to create the chart above using our new colour scheme we do the following:

1
2
3
4
5
PieChart chart = new PieChart(250, 100, PieChartType.ThreeD);
chart.SetData(new int [] { 60, 40 });
chart.SetPieChartLabels(new string [] { "FireFox", "IE" });
chart.SetDatasetColors(new string [] { "9913CE", "D3A4E5" });
string url= chart.GetUrl();

You can then use the URL directly in an HTML document or download the image locally using the following code:

1
2
3
4
WebRequest request = WebRequest.Create(url);
WebResponse response = request.GetResponse();
Image image = Image.FromStream(response.GetResponseStream());
image.Save("mychart.png");

Yep it’s that easy. The other charts can be created just as quickly in both the Google Chart API and also with the C# wrapper. So if you are looking to create good looking charts and don’t want to pay much (any) money I suggest that you check this out.

dynamic queries with LINQ

So, I’m a master C# programmer having been using it for a grand total of 4 weeks! My learning process began by ignoring the provincial and proverbial “Hello World” program, and instead I decided to learn the language features that are particular to C#.

As it happens, C# has quite a few modern language features that Java is sadly lacking (I won’t go into this here, but a comparison between the two is something I plan to blog about in the future). Along with many other new features, C# 3.0 gave us LINQ, which will be the topic of this post.

LINQ, I think, is reasonably straightforward to both understand and comprehend. Basic queries are extremely simple to create, and LINQ almost gives C# a functional programming feel. An example LINQ query is shown below:

from s in students where s.classyear == "2008" && s.courseyear == "2" select s;

That said, not everything is plain sailing with LINQ. The application I was writing required the where condition in my LINQ statement be dynamic. That is, a column specified in the where condition is only known at runtime. Achieving this functionality using the syntax shown above is not possible (which if you try it out I’m sure it will be clear why). To achieve this functionality the learning curve soon ramped up.

If you have searched on Google for dynamic queries in LINQ, you probably found these two articles, ScottGu’s Dynamic Linq and tomasp.net’s Building LINQ Queries at Runtime in C# – there were others, but few proved to be much help. ScottGu’s article was very good, and in essence solved my problem, but didn’t really help me understand the problem. The second article just went way over my head. Why? Well it came down to another new feature of C# that I didn’t know about, which was Expressions and Expression Trees, which, as it turns out, proved to be the key to solving this problem.

In addition to using the query syntax with LINQ (as shown above), you can also use query expressions. Query expressions are simply static methods that allow you to express a LINQ query. The example given above can be rewritten as follows using a query expression:

student.Where(s => s.courseyear == "2008" && s.classyear == "2");

On first looking at this example, it is still not obvious how runtime queries can be generated using query expressions. However, by taking a look at the type of the argument passed to Where helps us out:

Expression<Func<TSource, bool>> predicate

After some thought, this begins to make sense. The Where method takes a lambda expression whose parameter is of type TSource and returns a boolean. Therefore, surely we can create a lambda expression dynamically by building an expression tree for it? And yes, this is exactly what you are supposed to do.

C# experts are probably howling at me now and can’t understand why it took me so long to get there. However, this article is not aimed at you; it’s aimed at a newbie like me, and is simply trying to detail my thought process.

So now it was time to learn about Expression Trees.

Although I understood the concept of Expression Trees, they took a bit of getting used to – just don’t give up if you are struggling with it. In order to show how we can use Expression Trees to create dynamic queries, I will jump straight in with the code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
public Expression<Func<student, bool>> GetWhereLambda(string courseyear,
                                                      string classyear,
                                                      string myDynaColumn)
{
    ParameterExpression param = Expression.Parameter(typeof(student), "s");
 
    Expression courseExpr = GetEqualsExpr(param, "courseyear", courseyear);
    Expression classExpr = GetEqualsExpr(param, "classyear", classyear);
    Expression cond = Expression.And(courseExpr, classExpr);
 
    // This is where we create the expression for the dynamic column.
    // Obviously the value could have been dynamic as well but it 
    // saves a little visual complexity this way.
    cond = Expression.And(cond, GetEqualsExpr(param, myDynaColumn, "YES"));
 
    return Expression.Lambda<Func<student, bool>>(cond, param);
}
 
private Expression GetEqualsExpr(ParameterExpression param,
                                 string property,
                                 string value)
{
     Expression prop = Expression.Property(param, property);
     Expression val = Expression.Constant(value);
     return Expression.Equal(prop, val);
}

Seeing the code makes this look pretty simple, but there are quite a few concepts that a programmer new to C# has to get the hang of. However, let’s just step through the code – well one clause anyway, the rest follows from that.

Let’s look at how the clause (s.courseyear == "2008"), from out original where condition, converts to an Expression. We first create a parameter expression (Line 5), that is, the argument (parameter) supplied to the lambda expression (this parameter must be used for all sub-expressions that make up the lambda expression). Next, we need to create an expression to represent the right hand-side (s.courseyear) and left hand-side ("2008") of our clause. Since courseyear is a property of student (property as in C# property – think neat getters and setters) we create a property expression, i.e. prop = Expression.Property(param, "courseyear"), and as "2008" is really a constant, we create constant expression using value = Expression.Constant("2008"). Finally, we want to express the fact that the property should be equal to the constant in our query, we use the Expression.Equal(prop,value). That’s it. Essentially what expressions do is model our code as data.

Now it becomes obvious how we can represent a column in the where clause that is supplied at runtime – we simply create a property expression, as shown at Line 20, where property is the dynamic column name specified at runtime.

To combine the clauses in the where condition we simply use Expression.And, as shown in Lines 9 and 11. The resulting expression and the parameter is then used to create a lambda expression, which is in turn passed as an argument to Where, e.g.

Expression<Func<student, bool>> myLambda = GetWhereLambda("2008", "2", "active");
IEnumerable<student> filtered = students.Where(myLambda);

There you have it. It’s not that straightforward for those new to C# like me, but hopefully this article at least gives some code that can easily be used by people in the same position as I was.

generating a unique range of numbers

Often at work I find the need to generate a set of n unique integers in a specified range. In order to do this as efficiently and as easily as possible, I created a small C# class, which a colleague thought may be of general interest. Hence, I’m posting it here. I’m sure someone else has come up with a similar (or better) way to do this in the past, but I’m sharing my way regardless 😀 .

The code is shown below (and you download the C# class here):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
 class UniqueSetGenerator
 {
      private int[] store_;
      private int size_;
      private Random random_;
 
      public UniqueSetGenerator(int size, int start)
      {
          size_ = size;
          store_ = new int[size];
          random_ = new Random();
          PopulateArray(start);
      }
 
      private void PopulateArray(int start)
      {
          for (int i = 0; i < size_; i++)
              store_[i] = start++;
      }
 
      private int Delete(int pos)
      {
          int val = store_[pos];
          store_[pos] = store_[--size_];
          return val;
      }
 
      public int GetRandomNumber()
      {
          if (size_ <= 0)
             return -1;
 
          return Delete(random_.Next(size_));
      }
 }

I think it’s pretty easy to see how this works, so I will not go into it in much detail discussing it, but the sequence of figures below shows the basic operations.

First populate the array with the values.

First populate the array with the values.


Use the Random class to obtain our first random number - in this case 6. Now copy the value at position size_ in the array to position 6, and decrement size_. Note not we don't actually delete anything from the array.

Use the Random class to obtain our first random number - in this case 6. Now copy the value at position size_ in the array to position 6, and decrement size_. Note: not we don't actually delete anything from the array.


We then use the Random class to generate another random number in our reduced range - in this case 3. We then copy the value at position size_ (i.e. 9) to position 3, and then decrement the size_ count as before. This process continues until size_ is reduced to 0.

We then use the Random class to generate another random number in our reduced range - in this case 3. We then copy the value at position size_ (i.e. 9) to position 3, and then decrement the size_ count as before. This process continues until size_ is reduced to 0.

So when would you require something like this? Well, say you need to generate unique random IDs with values 1 to 10, then you can use the class as follows:

1
2
3
4
5
6
7
8
    UniqueSetGenerator uniqueSet = new UniqueSetGenerator(10, 1);
 
    for(int i = 0; i < 9; i++)
    {
        int id = uniqueSet.GetRandomNumber();
 
        // Now do something with the id....
    }

For the moment I have only included a C# version but I will update this post with a Java version soon. Hope some of you find this useful.