More than 1GB served in May

by Tobias Hertkorn on June 18th, 2007

Of course I had to test my mean value calculator. So I did some minor modifications and made it parse last month’s apache logfiles. And for the first time I had more than 1GB of data served solely by this blog. This blog served 92000 requests (all kinds of http requests) to 9500 individual IPs (sadly only about 2500 show no sign of being a spammer). There were about 12800 requests for the feed of the blog from 750 individual IPs.

So trying to filter out spammers this boils down to 3000 real blog pages served to 2200 unique (human) visitors from 93 different countries in May. With about 20 real subscribers to the feeds. Most visitors are from the US (30%), then Great Britain (7%), Germany (6%) and India (6%). There even was a visitor from exotic places like Vatican City and Ghana.

The largest amount of hits come from Google with the keyword combination “google toolbar iceweasel” followed by queries related to hbm2net and nhibernate. By far the weirdest search was “what is meant by website in asp.net”, I’m not even kidding. Pretty vague was “codes to look at stuff” and the ones that made me literally laugh out loud was “scifi operating system” and “tobi march”, the first one for obvious reasons, the second one, because the context of the search was probably sooo different from what the surfer found on my blog. The sadest one was “tobi smells”. No I don’t. :( Because I am (or at least my site is) “way too hot”. ;) The one search that got me a little scared was “pro and con of using fear, uncertainty and doubt”. Who is searching for pros on that topic?

Other than that:
The largest traffic magnets on this server are by the way SiKoRe a German site about improving your skills in mental arithmetic and latex2html the main repository of the LaTeX to HTML converter. Great jobs guys, but watch out. With the kind of increase my blog is showing over the last couple of months, I’ll get you soon. ;)

I have to thank you, my dear reader, for that. I am surprised that you like my blog (google analytics even reveals that some of you do even come back! ;) ). I hope you keep enjoying it in the future as well.

Post to Twitter Tweet this

June 18th, 2007 12:52 am | Comments (0)

Improving on yesterday’s mean value calculator

by Tobias Hertkorn on June 17th, 2007

There are still some bugs in yesterday's bash script. For example there is a nasty condition in the awk part. What happens if the mean value of an empty file is calculated? Awk of course complains "fatal: division by zero attempted".

Improvement:

BASH:
  1. awk '
  2. BEGIN {
  3.   FS=","
  4.   total = 0
  5.   count = 0
  6. }
  7. {
  8.   total = total + $2
  9.   count++
  10. }
  11. END {
  12.   if(count> 0) { print total/count } else { print "n/a" }
  13. }'

And there are of course some interesting side effects in the command parsing snippet. But that's for another day, I guess. Because improving that will require some real array voodoo. :-)

Post to Twitter Tweet this

June 17th, 2007 6:38 pm | Comments (2)

How to create a userfriendly bash script (command line parsing)

by Tobias Hertkorn on June 16th, 2007

Well, I had to do some very basic calculations based on simple csv files. I had to pick the second field and calculate the overall mean in a file of that field. There are some very good awk tutorials out there, but I was very disappointed about how little they talk about how to surround the awk command with a little bash sugar to produce a small reusable tool. So I just wanted to quickly point out how to something like that WELL. The basic awk algorithm is very simple, but you really should surround that simple awk command with some basic tests. It is pretty important, at least as far as I am concerned, that tools are always blessed with a nice command line parser, that it allows for direct file referencing and piping as well, if an error occurs please make it write to stderr and make it returns correct exit status codes. It's just good practice. Granted, bash command line parsing is not too much fun, but there are a couple of tricks, but the exit codes are so easy to deal with, just remember putting in one line.

I'll just throw out the code, so look at it and maybe you can identify all the things that went into the surrounding sugar:

BASH:
  1. #!/bin/bash
  2. do_usage(){
  3.   echo>&2 "Usage: `basename $0` [-|-f file|--file file] [--] [file ...]"
  4.   exit 1
  5. }
  6. test_if_last_argument() {
  7.   if [ $# -gt 0 ]; then
  8.     do_usage
  9.   fi
  10. }
  11.  
  12. filename=
  13. while [ $# -gt 0 ]
  14. do
  15.   case "$1" in
  16.     -f|--file) filename="$2"; shift;;
  17.     --) shift; break;;
  18.     -) filename="-"; shift; test_if_last_argument "$@"; break;;
  19.     -*) do_usage;;
  20.     *) break;; # terminate while loop
  21.   esac
  22.   shift
  23. done
  24.  
  25. awk '
  26. BEGIN {
  27.   FS=","
  28. }
  29. {
  30.   mean = mean + $2
  31.   count++
  32. }
  33. END {
  34.   mean = mean / count
  35.   print mean
  36. }
  37. ' "$filename" "$@"
  38.  
  39. exit $?

It is an excellent example of how there usually 56.7% of the lines are just there for maintenance and only 43.3% actually do something useful. But always remember: 42.7 percent of all statistics are made up on the spot.

I'll explain why I put those maintenance lines in, why it's "$@" and not $@, etc ... tomorrow.

Post to Twitter Tweet this

June 16th, 2007 11:43 pm | Comments (0)

Trying to enhance ObjectDataSource

by Tobias Hertkorn on June 1st, 2007

Edit: Please be sure to check out the comments! Eilon Lipton the author of ObjectDataSource was so nice and did talk to me about my problem. Awesome! He even went as far as creating a blog post Using ObjectDataSource to do the dirty work for your custom data source over at his blog about it. But first enjoy the original story posted on Sept. 9th 2006:

The ObjectDataSource is a great new tool in ASP.NET 2.0. But unfortunatelly its primary intended way of using it, is by using static methods or methods of objects that get created using the default constructor. But this is very limiting when building complex websites. Especially complex websites that should be maintainable. ;) That means it is not easy using the ODS when employing software cells, a Model View Presenter approach, capsulation of logic, etc. Especially MVP suffers from a considerable amount of work-around-uglyness, when using the standard ObjectDataSource. Sure, it is possible to override the default object creation process by attaching to the ObjectCreating event.

XML:
  1. <body>
  2.     <form id="form1" runat="server">
  3.     <div>
  4.       <asp:ObjectDataSource ID="ODS1"
  5.             runat="server"
  6.             OnObjectCreating="ODS1_ObjectCreating" />   
  7.     </div>
  8.     </form>
  9. </body>

and in the code behind:

C#:
  1. protected void ODS1_ObjectCreating(object sender, ObjectDataSourceEventArgs e)
  2. {
  3.     e.ObjectInstance = presenter.getAdapter();
  4. }

Sure, that could be a solution - but I'd rather do something like this:

XML:
  1. <body>
  2.     <form id="form1" runat="server">
  3.     <div>
  4.       <myDS:AdapterDataSource ID="ADS1"
  5.             runat="server"
  6.             AdapterProvider="presenter"
  7.             AdapterProviderMethod="getAdapter"
  8.             AdapterSelectMethod="Select"
  9.             AdapterTypeName="TestAdapter" />
  10.     </div>
  11.     </form>
  12. </body>

Ha, EAAAAAAASY to do, right? Just override the method onObjectCreating of the base class ObjectDataSource in the new class AdapterDataSource. Well, if you look at the specs you will realize that the Event is not actually raised in ObjectDataSource - just made public there. The actual event is raised inside ObjectDataSourceView. But still no problemo, right:

C#:
  1. public class AdapterDataSourceView : ObjectDataSourceView
  2. {
  3.   private AdapterDataSource m_owner;
  4.  
  5.   public AdapterDataSourceView(AdapterDataSource owner, string name, HttpContext context)
  6.     : base(owner, name, context)
  7.   {
  8.     m_owner = owner;
  9.   }
  10.   protected override void OnObjectCreating(ObjectDataSourceEventArgs e)
  11.   {
  12.     // Let the ADS decide if we have to do some magic here.
  13.     if (m_owner.HasCustomDataBinding)
  14.     {
  15.       e.ObjectInstance = getCustomBinding();
  16.     }
  17.     base.OnObjectCreating(e);
  18.   }
  19.  
  20.   private object getCustomBinding()
  21.   {
  22.     throw new System.Exception("The method or operation is not implemented.");
  23.     // e.g. return m_owner.CustomDataBinding;
  24.   }
  25. }

So now, all we have to do is override the one method of ObjectDataSource in the AdapterDataSource that generates those ObjectDataSourceView instances, right... WRONG. No method seems to be responsible for creating the view.

Using the reflector I found out what was really happening behind the scenes.

C#:
  1. private ObjectDataSourceView GetView()
  2. {
  3.   if (this._view == null)
  4.   {
  5.     this._view = new ObjectDataSourceView(this, "DefaultView", this.Context);
  6.     if (base.IsTrackingViewState)
  7.     {
  8.       ((IStateManager) this._view).TrackViewState();
  9.     }
  10.   }
  11.   return this._view;
  12. }

The method used is called getView() inside the .NET Framework provided by Microsoft - and it is private! Yeah, baby. :) All those protected methods inside the ObjectDataSource class are just for show - because the main step in the process of creating the actual view can not be overridden.

But I wasn't going to give up that easily. How about overriding all methods that depend on GetView(). That way I could retire the private GetView() and great my own view creation algorithm. So full of hope I fired up the Analyser. But when I saw the following I started sobbing quietly:

ObjectDataSource GetView Disaster

Well, to make a long story short - I did try to do the overriding and stuff - but unfortunatelly those methods sometimes use Attributes that are internal and all kind of crazy stuff that makes you want to beat the programmer that did put that little innocent "private" in front of GetView(). ;)

Because it is 3 in the morning and I am really tired I did use this shortcut:

C#:
  1. public AdapterDataSource()
  2.   : base()
  3. {
  4.   Type baseType = typeof(ObjectDataSource);
  5.   FieldInfo fi = baseType.GetField("_view", BindingFlags.Instance | BindingFlags.NonPublic);
  6.   AdapterDataSourceView sourceView = new AdapterDataSourceView(this, "DefaultView", this.Context);
  7.   fi.SetValue(this, sourceView);
  8.   if (base.IsTrackingViewState)
  9.   {
  10.     ((IStateManager)sourceView).TrackViewState();
  11.   }
  12. }

UUUUUUUUUUUUUUUUUGLY! And it breaks when using Mono - because there it's a private property called DefaultView. And the best part about this: There actually is a PROTECTED method called GetView(string viewName). But that does not get used, probably because it returns simply a DataSourceView instead of a ObjectDataSourceView...

C#:
  1. protected override DataSourceView GetView(string viewName)
  2. {
  3.   if ((viewName == null) || ((viewName.Length != 0) && !string.Equals(viewName, "DefaultView", StringComparison.OrdinalIgnoreCase)))
  4.   {
  5.     throw new ArgumentException(SR.GetString("DataSource_InvalidViewName", new object[] { this.ID, "DefaultView" }), "viewName");
  6.   }
  7.   return this.GetView();
  8. }

I just wonder why nobody considered using:

C#:
  1. protected readonly string DEFAULT_VIEW = "DefaultView";
  2.  
  3. protected override DataSourceView GetView(string viewName)
  4. {
  5.   if ((viewName == null) || ((viewName.Length != 0) && !string.Equals(viewName, DEFAULT_VIEW, StringComparison.OrdinalIgnoreCase)))
  6.   {
  7.     throw new ArgumentException(SR.GetString("DataSource_InvalidViewName", new object[] { this.ID, DEFAULT_VIEW }), "viewName");
  8.   }
  9.   if (this._view == null)
  10.   {
  11.     this._view = new ObjectDataSourceView(this, DEFAULT_VIEW, this.Context);
  12.     if (base.IsTrackingViewState)
  13.     {
  14.       ((IStateManager) this._view).TrackViewState();
  15.     }
  16.   }
  17.   return this._view;
  18. }
  19.  
  20. private ObjectDataSourceView GetView()
  21. {
  22.   if (this._view == null)
  23.   {
  24.     return GetView(DEFAULT_VIEW) as ObjectDataSourceView;
  25.   }
  26.   return this._view;
  27. }
  28.  
  29. private ObjectDataSourceView _view;

*SOB*

Update:
Marc Brooks pointed out that there is a bugreport on this issue. Check the comments to find out how that "worked out".

Post to Twitter Tweet this

June 1st, 2007 7:01 pm | Comments (8)
Tobi + C# = T# - Blogged blogoscoop