Dawn of the Geeks

Using the MySQL Odbc 5.1 connector I came across a strange error: memory allocation error. It’s like the “pc load letter” of MySQL. In my case the issue was that certain parameters were being set to null values. Once I defaulted the strings to an empty string rather than null the next error to pop up was that the parameters count wasn’t the expected amount. Meaning I had too few or too many parameters added with value. And once that was corrected it worked just fine.

So apparently in some cases Odbc doesn’t like null parameter values. My guess is that the table column was set to not allow null and so it kicked back an obcure error. Either way it works now.

Tweemotional.com was not doing so well. It had too much information in the database which was also missing an index on the date causing the SELECT to do a file sort on millions of rows.

So I took the opportunity to flush the database since raw tweet data that is more than 24 hours old is ignored anyway and added in the index.

However, I also plugged in Memcached support. With Memcached you can do the query on the database once, take the result and stuff it in memcached. The next time you need the data you simply query memcached which returns the result instantly.

So now when the backend scripts for Tweemotional.com are gather information from Twitter they also update the Memcached versions of what the site needs to display the data. So now the site itself runs very quickly since it’s not hitting the database nearly as much.

If you want to make use of Memcached for your site you’ll need pretty much any computer you can get. I have it running on a PIII 533Mhz system with 512MB of memory. Obviously if you have a site that could make use of more space to store cached objects then you’ll need a system with more RAM or diskspace. For best performance you want enough memory that memcached doesn’t have to pull information from the disk before sending it over.

There are Windows binaries available but they are for an older version. If you want to run the latest version you’ll probably want to just use some version of Linux. I prefer Ubuntu. The 8.04 version installed and ran just fine on my system. I turned off the graphical flair for the GUI and it runs rather smoothly.

There are existing libraries for Memcached for PHP and C# as well as other languages. Enyim.Caching is what I use with C#. I then wrote a simple static class wrapper around that so I can just use memcached.get, memcached.set and memcached.remove and not have to deal with any initialization.

Another reason for using a wrapper is that memcached doesn’t like keys with certain characters. To avoid this problem the wrapper automatically MD5’s the key I pass in before it passed it along to memcached. In my code I can then use whatever key I want.

The other thing you have to remember with C# is that you must explicity make your classes Serializable or they cannot be cached through memcached. I don’t believe PHP has this issue. I’m not sure if there’s anything in PHP that isn’t serializable by default.

If you’re running any sort of website that is doing some heavy lifting with queries it would be to your benefit to take a look at memcached to see what it can do for you.

There are two key things you need to do in Superheroes Alliance on Facebook: win fights and complete missions. Completing missions in a reasonable amount of time requires having plenty of energy. Winning battles requires attack strength with a supporting alliance and quality items for each person in your team including yourself.

Defense is used mainly when defending yourself against other people’s attacks. In the event you lose a battle you lose HP and money. If you don’t keep cash on hand there’s no financial incentive for others to attack you and so they don’t. They hit you once and if they win they see they get nothing for it so they move on to the next target. The first step to avoid being attacked is to keep your money in the vault. It costs you 10% but that’s better than losing it all from being attacked constantly. Don’t worry about defense points and instead focus on increasing attack points.

HP are your “hit points.” You lose them every time you get hit. The only thing you need to worry about is losing more than you have in a single hit. Once you go below 25 you can no longer be attacked. This is why, at least for awhile, you don’t need to waste points increasing your HP.

You need one stamina point for each attack. You don’t really need to increase this since you can always go back and attack someone again after recovering your stamina and HP.

Completing missions gets you money, better items and experience which moves you through the levels. You also get extra points for the lab. Use them to get hired heros. The more people you have on your team the better chance you have of winning battles.

The second and most important thing to do to avoid being attacked is to win more battles than you lose. Generally people are going to attack the weaker players so if you win more than you lose other players will for the most part avoid you.

So above all else, increase your attack points the most, then energy. As you progress through the levels you can start to increase other stats as well. But you need to win battles and complete missions before you can really start rounding out your character.

I used to play Heroes vs Villians on Facebook until I discovered Superheroes Alliance. Superheroes Alliance is a very well polished game that plays much like HvV but looks significantly better and has better features. For example you can see how many wins and loses another player has which makes it easier to figure out who you’d like to attack. You can also battle rivals which means you can keep track of how many wins and loses you have against specific players and keep battling them if you beat them most of the time.

I’ve been looking for tips and tricks for this game and havn’t found any so I’m going to offer a few.

Prior to the complete reworking of missions they were pretty much useless. Now you need them to get better sidekicks. However, your focus, especially early on, should be only on attack and defense. The real money and experience comes not from missions, but from attacking and winning battles against other players.

Money is important in this game. If you want better sidekicks you needs income to support them. If an abilitity is available to get more money from winning battles, take advantage of it. Use most of your skill points to magnify attack and defense where possible but don’t neglect the money train.

Only buy land and property in blocks of 10. The price goes up significantly with each purchase so the more you buy all at once the more money you save. To save the most money buy 10 blocks of land when you can. Buy no more than 10. You will start earning money on the land. Then buy 10 blocks of property to go on the land when you can. The price of the land will drop back down to the lowest price and you can repeat the process.

Keep track of the amount of money you earn vs the cost of the land + property. You want to maximize your return so buy the land+property combo that results in the greatest earnings for the cost no matter how little the earnings are. It all adds up quickly when you consider the amount you will earn over several days.

By the time you own 30 of a property it takes about a week to recoup the cost. When you’re at that point it’s time to start saving up for the big ticket items.

Again high defense and attack stats and top of the line sidekicks, you can quickly bring in millions by battling.

Use the vault often. Yes it costs you 10% but that’s significantly less than it costs when someone beats you in a battle. I typically vault the money until I’m within 5-10 million of my goal. Once I have the last 5-10mil on hand I withdraw just enough to cover my purchase. Make sure you’ve regenerated all your health before sinking all the money you have into your purchase.

Once you have millions of dollars coming in every hour it’s time to stop focusing so much on making money and focus more on completing missions. That means using some of your skill points to up your maximum energy.

As you progress in the game the goal is to depend less on your character and more on your alliance and sidekicks. You don’t need maxed out attack and defense because your multipliers, sidekicks and alliance take care of that.

The Emotional State of the Internet is a website which analyzes Twitter messages and calculates how the internet is feeling. It’s updated every 10 minutes. You can also see a historical chart of the current top emotion.

It’s interesting to see the difference between what emotion people twitter about the most in various languages. French speakers talk about how lonely they are. Germans by a wide margin talk about “angst” the most. While Google Translate tells me it means “frightened” in English, according to Wikipedia it more accurately translates to “anxiety.” This is why I keep the original word listed and put the English translation in parantheses.

Italians tend to talk most about being “triste” which means sad while Spanish speakers talk by a slightly wider margin about being happy.

Maybe Spanish speakers are happier than everyone else or maybe they use Twitter mostly to express happiness and tend to keep other emotions to themselves.

If you want to what people are getting emotional about you can click on “keywords” at the top of the page. That will list all the current keywords for all the various emotions the site is tracking.

Solid state drives are finally becoming large enough that computers can be built without a conventional hard drive. Any modern OS can be installed on a solid state drive now. However, they’re very expensive and can’t hold as much as the conventional drives. The basic idea is that you use a solid state drive for your OS to speed up your system but continue using large conventional drives for all your personal data. It’s generally a good idea to keep your personal data on one drive and your OS on another regardless of what type of hard drives you use.

What I’m finding flash drives to be very useful for is running web sites off of them. I’ve been running my own servers for years and end up having to buy new hard drives every year which gets expensive. I realized that the majority of my web-sites are essentially static and they’re small.

So for $5 I can store all my web-sites on a 1-2GB flash drive. If the drive dies I can replace it for $5 instead of $35-100.

The advantage of using a flash drive is that the sites load much faster. The hard drive is no longer a limiting factor for the site itself. Just the database which of course still uses a regular hard drive since flash drives die quickly with heavy writing.

My server used to have a hard drive for the OS and web apps, the database, the logs and the web-sites themselves. I stopped bothering with logging since it’s too much useless information and there are better ways to log errors and visits without being limited to the server app (Apache or IIS). So now my server is down to two conventional drives and as many flash drives as I need.

Putting the “geek” in Dawn of the Geeks are two servers I have set up.  The first server is running Windows XP with Apache, MySQL, PHP and Subversion.  Recently I began developing .Net web sites using C#.   Namely the bank site which is used for managing and tracking personal finances.  Windows XP only allows a single web-site to run and only has IIS 5.1.  Since I have a copy of Windows Server 2003 I decided to put it to use and installed it on another server.  That server just has Windows 2003 with II6.

Now the issue is that all port 80 requests go through Apache. So the first step was to install mod_proxy and mod_proxy_html. Now I can point various hosts to the second server. However mod_proxy doesn’t pass along the original host header. It was sending it to the internal IP. Well, IIS 6 needs to know what host it’s serving for as well if you want to run multiple sites.

The trick is to modify your hosts file on the Apache server so that the host name maps to the internal server.

The flow is

bank.dawnofthegeeks.com hits the DNS server and points to the public IP of my router. The router sends the port 80 request to the apache server. The apache server sees that it’s dealing with bank.dawnofthegeeks.com and reverse proxy’s it to bank.dawnofthegeeks.com, however when Windows XP tries to resolve that host name it gets the internal IP from the hosts file rather than the public IP of the router from the DNS servers. So the request gets sent to the IIS server which sees the hostname bank.dawnofthegeeks.com and serves up the correct web back through the proxy.

Through the magic of the hosts file Apache reverse proxies a host to the same host.

This is log.js that goes in a central location

log.js

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

if (typeof(km_scripts) == 'undefined') 
	var km_scripts = new Object();
 
km_myclass_import('http://js.dawnofthegeeks.com/hitlog.php?REQUEST_URI=' + document.location + '&HTTP_REFERER=' + document.referrer);
 
function km_myclass_import(jsFile) 
{
	if (km_scripts[jsFile] != null) 
		return;
	var scriptElt = document.createElement('script');
	scriptElt.type = 'text/javascript';
	scriptElt.src = jsFile;
	document.getElementsByTagName('head')[0].appendChild(scriptElt);
	km_scripts[jsFile] = jsFile;
}

This line of HTML goes at the bottom of every page that I want to track

1

<SCRIPT LANGUAGE="JavaScript" SRC="http://js.dawnofthegeeks.com/log.js"></SCRIPT>

hitlog.php does the hard work of logging the various information to a MySQL database. I can then go to my stats page and see the unique visitors and page view count for every domain that line of HTML appears on. I could break it down by pages as well but for now I’m not that interested in that level of detail in my reports.

In order to avoid having massive unmanagable tables (GoDaddy limits you to 200MB per database) each month has its own table. If I’m running out of space in the database I can export the older tables and delete them from the GoDaddy servers.

hitlog.php

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97

 
// define: $dbhost, $dbuser, $dbpass, $dbbase;
 
function ConnectToJS()
{
	global $dbhost, $dbuser, $dbpass, $dbbase;
 
	$db = mysql_connect("$dbhost", "$dbuser", "$dbpass");
	if(mysql_select_db("$dbbase",$db))
		return $db;
	return 0;
}
 
$db = ConnectToJS();
 
$table = "logs_" . date("Y_m",time());
 
$sql = "
CREATE TABLE IF NOT EXISTS `$table` (
  `id` bigint(20) NOT NULL auto_increment,
  `hash` varchar(255) NOT NULL,
  `ip` varchar(255) NOT NULL,
  `ip_int` bigint(20) NOT NULL,
  `ip_host` varchar(255) NOT NULL,
  `ip_country_id` int(10) NOT NULL,
  `created_at` datetime NOT NULL,
  `path` varchar(255) NOT NULL,
  `referer` varchar(255) NOT NULL,
  `referer_host` varchar(255) NOT NULL,
  `query_str` varchar(255) NOT NULL,
  `agent` varchar(255) NOT NULL,
  `host` varchar(255) NOT NULL,
  PRIMARY KEY  (`id`),
  UNIQUE KEY `hash` (`hash`),
  KEY `host` (`host`),
  KEY `ip_country_id` (`ip_country_id`),
  KEY `referer` (`referer`),
  KEY `agent` (`agent`),
  KEY `created_at` (`created_at`),
  KEY `referer_host` (`referer_host`)
) ENGINE=MyISAM 
";
 
mysql_query($sql,$db);
 
 
$ip = $_SERVER['REMOTE_ADDR'];
$ip_host = gethostbyaddr ( $_SERVER['REMOTE_ADDR'] );
$path = $_REQUEST['REQUEST_URI'];
$parts = parse_url($path);
 
$ip_int = explode(".",$ip);
$ip_int = $ip_int[0]*256*256*256 + $ip_int[1]*256*256 + $ip_int[2]*256 + $ip_int[3];
 
$sql = "
	SELECT
		id
	FROM
		ip_to_country
	WHERE
		$ip_int >= ip_start AND $ip_int <= ip_end
";
$res = mysql_query($sql,$db);
$r = mysql_fetch_assoc($res);
$ip_country_id = isset($r['id']) ? $r['id'] : 0;
 
$created_at = time();
$referer = $_REQUEST['HTTP_REFERER'];
$ref_parts = parse_url($referer);
$referer_host = $ref_parts['host'];
$query_str = $ref_parts['query'];
 
$agent = $_SERVER['HTTP_USER_AGENT'];
$host = $parts['host'];
$hash = md5($ip . $created_at . $path . $referer . $agent . $host);
 
 
 
$sql = "
INSERT INTO 
	$table
SET
	hash = \"" . mysql_real_escape_string($hash) . "\",
	ip = \"" . mysql_real_escape_string($ip) . "\",
	ip_host = \"" . mysql_real_escape_string($ip_host) . "\",
	ip_int = \"" . mysql_real_escape_string($ip_int) . "\",
	ip_country_id = \"" . mysql_real_escape_string($ip_country_id) . "\",
	created_at = NOW(),
	path = \"" . mysql_real_escape_string($path) . "\",
	referer = \"" . mysql_real_escape_string($referer) . "\",
	referer_host = \"" . mysql_real_escape_string($referer_host) . "\",
	query_str = \"" . mysql_real_escape_string($query_str) . "\",
	agent = \"" . mysql_real_escape_string($agent) . "\",
	host = \"" . mysql_real_escape_string($host) . "\"
";
 
mysql_query($sql,$db);

A-Star (A*) Implementation in C# is an excellent implementation of A* in C#. It plugged neatly into Bunnies 3.0 which will soon have a sequal “Zombies.” The AI in Bunnies consists of stupid bunnies that just bounce around and multiply and slightly more intelligent enemies that will come at you and shoot at you if they see you. In “Zombies” you can run but you can’t hide. The game will be slow moving but difficult to kill hordes of enemies with limited ammo and various weapons. The A* algorithm is being used by the zombies to always keep them moving towards you no matter where you go.

This isn’t my first use of the A* algorithm and the flaws with it are still there. My first use of A* was for a simulator that had tanks and flying vehicals moving around complex 3D terrain that was created using DEM data (geological survey data of real life locations). One of the test paths took a helicopter over Mount Saint Helens. The helicopter would fly up the side of the mountain at the maximum pitch, fly down into the mountain, fly up out of the mountain and then fly down the side of the mountain to the destination.

The reason it did this is because A* always wants to be close to the final destination which was at the base of the other side of the mountain. So if it got a chance to lower the altitude to be close to the base it did. But that’s obviously not the optimum path. The fix for flying vehicals is to find the farthest point in the list of destinations which can be seen with a simple line of sight check. So from the base of the mountain the helicopter can see the top of the mountain so it heads straight there. From there it can see the other side of the rim so it flies straight there. And from there it can see it’s destination so it flies straight there. A path that was dozens of points is reduced to less than 10.

The technique is basically just doing intelligent post processing on the A* path to cut out unnecessary steps. Land vehicals had limits on the angle of the terrain they were going over. Sometimes A* would come up with tangled paths that didn’t go anywhere. So to fix that I found the farthest point along the A* path that the vehical could see and travel to without violating the steepness rule. That easily cut out knots that A* had created and simplified paths significantly.

Because the Bunnies Engine deals with flat terrain the line of sight algorithm can be used on the zombies to simplify their movement. Once I start putting together an actual tutorial on A* I’ll give visual examples of how post processing helps clean up A*.

The original software rendering location will remain up for now while I slowly transition each of the lessons to the new Real Time Software Rendering Blog.

I found that it was quite the pain to update the old site since it was all static HTML and any syntax highlighted code was highlighted by hand. WordPress has a handy dandy plugin that automatically does the syntax highlighting. So this move will hopefully motivate me to continue adding new lessons on more topics.

There is certainly no shortage of lesson ideas that I’d like to cover.