I’d been quietly following AppFog’s progress since I received a PHPFog beta invite a year or two ago. At the time I hadn’t had a reason to give their service a try since I wasn’t a PHP guy. However, spurred by the allure of a coding contest and the large number of platforms AppFog supports, I finally had a reason to poke around!

Anticipating that I’d be up against stiff competition I chose to place some additional requirements on my contest submission. As the AppFog Blog recently showed I wasn’t wrong about the competition. My hope was that the extra work I put into the submission would grant me at least an honorable mention.

What I didn’t expect was that I’d end up winning the competition!

I am honored that my code was chosen as the winning submission. As a small show of thanks, I’ve decided to detail my experiences with the AppFog beta as well as annotate the de-obfuscation of my code step-by-step.

An AppFog First Impression

My first concern was getting an AppFog instance configured on which to develop. While I do have a little bit of experience dealing with shared hosting and VPS services, I have only minimal exposure to PaaS providers such as AppFog. At first I was worried about the work required to configure an application, but AppFog’s simple interface had me finished with the process before I knew I had started!

After cloning an example project published by AppFog on github, installing the AppFog gem, and running a deploy I was done! Zero to functioning application in almost exactly 2 minutes!

Thanks to the simple in AppFog deployment process I was able to concentrate entirely on my code, and not my hosting.

Some Speed Bumps

Unfortunately I can’t say that my experience with AppFog was completely problem free. During development I ran into occasional trouble with deployment. Applications that ran perfectly fine locally started timing out regularly during staging. When they staged they would fail to start. A few of these problems were of my own making and were easily resolved by checking the support area of the AppFog site.

For the more difficult problems, the AppFog support team met or exceeded the customer service I’d grown accustom to from both Dreamhost and Linode. With the help of the AppFog support team I was able to fix my deployment problems and submit my solution. I was especially impressed by their responsiveness given I hadn’t paid them a cent!

Enough about problems… on to the code!

Creating a Contest Submission

Now, I love obfuscation just as much as the next guy, but I wanted my submission to be slightly more than just a block of funny looking text. I wanted my submission to have a few extra (admittedly simple) levels of trickery involved. In order to satisfy this urge I added the following extra personal submission requirements:

1. The code must be pretty.
2. The code must be obfuscated in at least two separate ways.
3. The final “Happy Birthday OpenStack” text must exist in some way outside of the source code itself.

In order to understand how I achieved these objectives lets walk backwards through the obfuscation steps to reveal the original source code.

Note: What follows is a step-by-step analysis of the code itself, performed through the lens of someone de-obfuscating my submission. The contents of this analysis are targeted at someone with minimal Ruby experience. If you feel you understand Ruby you may get more enjoyment out of attempting the de-obfuscation yourself!

Understanding the Obfuscation

None of the methods of code obfuscation I used are terribly complex. The full submission is plenty pretty with ASCII art declaring my affection for the people running the contest.

The ASCII art itself was created with the help of an open source tool called jp2a by Christian Stigen Larsen. The generated ASCII was then used to format the resulting block of obfuscated code. This method allowed me to obfuscate the code first, then tweak the ASCII art as I saw fit.

Despite how complicated it may look my submission boils down to the following four lines of code:

1
2
3
4

c = %w(A bunch of characters with lots of whitespace).join
z = 'Znqr_ol_Wbr_"pbqrewbr"_Onhfre'
XYZZY = __FILE__
send(%w(ale lava).reverse.join.split('').uniq.join.reverse.to_sym,c.split(/0x/)[1..-1].map {|b|(b.to_i(16)>>2).chr}.join)

Despite separating the lines of code it is still plenty hard to understand what it is doing. Lets take it line by line:

1

c = %w(A bunch of characters with whitespace).join

The above uses Ruby’s quoted array syntax to turn a bunch of whitespace separated strings into an array of values.

1
2
3
4
5
6
7
8
9
10
11
12
13

#The whitespace separated tokens become strings in the array
%w(one two three) # => ["one", "two", "three"]

#Both using an uppercase W will function the same
%W(one two three) # => ["one", "two", "three"]

#If your tokens need to include parentheses the quoted character can be almost any
#special character. If the special character is one of {, [, (, or < then the end
#character will be the matching closing delimiter. For other characters it is just
#the same delimiter
%w[one two three] # => ["one", "two", "three"]
%w!one two three! # => ["one", "two", "three"]
%w|one two three| # => ["one", "two", "three"]

This means that the code takes the humongous block of gibberish and turns it into a huge array of tokens. The join method is immediately called on the response, meaning the variable c contains a large string of characters.

The second line is a red-herring rot13 representation of the phrase Made_by_Joe_"coderjoe"_Bauser" which can be safely ignored.

The third line:

1

XYZZY = __FILE__

Does nothing more than store the value of __FILE__ (which represents the name of the currently running file) into a constant variable named XYZZY.

The last line is the meat and potatoes of this file. It has the job of interpreting the huge block of text! If we create some temporary variables we can see what is going on a little easier.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

# We've seen %w() once before already so we know what that does
# the rest of the methods operate on that value producing the symbol
# value :eval
arg1 = %w(ale lava).reverse.join.split('').uniq.join.reverse.to_sym

# The second argument gets its value from the C variable defined at the top of the file.
# C is first split by the string 0x into a new array. The first element of the array is
# omitted using the range [1..-1] and map is applied to the new array of values.
arg2 = c.split(/0x/)[1..-1].map do |b|
  b = b.to_i(16) # Each value of the array is converted from a hex value into an integer
  b = b >> 2     # Right shifted by two
  b.chr          # And the integer value is converted to an ASCII character and returned
end

# This newly formed array of ASCII characters is joined together!
arg2 = arg2.join

# Now that we have our two arguments, they're used in the Object#send method.
# Object#send sends a given message to an object (in our case identified by the symbol
# :eval stored in arg1) providing the arguments which follow.
send( arg1, arg2 )

Now we know what the obfuscated code is! It’s a series of ASCII characters, left shifted twice, converted to hexadecimal, and separated by the string ‘0x’. It is being used as the value to send(:eval,string) which means it must be more ruby code!

Why did I do this? I had started with simple hex conversion, but I found I needed a larger canvas on which to draw my ASCII art, so the left shifting increased the hex values enough that I had a little more text to work with!

We Have to Go Deeper

Now that we know how the code is interpreted, we print it by replacing the code creating the :eval symbol with the symbol :print and the application will dutifully output the vanilla code!

The output of which is as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

require 'zlib'
require 'base64'
require 'coderay'
require 'uri'
require 'sinatra'

module Wizardry
  def self.do_some_magic( request )
      iywRFqowCSzrNe = [
          "==geeZ6bD0nS1mp8AGeHt2VR+Tmyq3mD7rt5jLg7\nu9dxALYH4m/+GLeH6FKh4IlsCvFaCqFMQvZg0jN6YjuY2g/PR8zCu2FvLkE/\n/jX629hL4wfVvJOm6YGkvLne1Gn/xHgYFnhGzxYxzuzU14EScm+/Jv8ErvIB\nZcdx1ftynZWYIyxucKb9tVdqU1aR9saTVwak7lKC0jKS7bHEHzLfNbuZehgs\npu7/KtYaZKNbwezVdLdWK9iWRuE2CuIwDRcFklLJ9sUF5/ehQEzALF0jdpNe",
          "=oGq17wPIy0sZJ/JgLjZvX7vpr/D7vtCK+C0/Nr3fOv6\nQpIN7OCFRp3M4lxUwHyWOdTJBXHf3phtezdqJtPv00BFFJF+Z/SnLmANQeID\n6sl8Qf5oiuQ97DvchW+aqUT2k3Nn5e/8mj5lGx4j6q/Uze77MGmDe9R7OPey\n7/N0ZSimTZE9lpOmJzS7mgpqoRdsJLuLbNAmTnwnZikMjnuCl/pTainokOWE\nRuxORDYibKrh8cMQ7691WoTRaTekD2FwP598I7LRXbHmIWV6jciXQMcQ0IU8\ng6q22SaDJZUEq0LEN2FG2fWONOFFWw94DE9WVasFQsappUZYsLVKXZdpNanq\nIYdgWUNq5EAWWBtlHmIaXflSSwdENagIKQQim+dgdsmpPwUM3zr3hxQhIo4s\n9+frt2amxYSza4o7lIq02HLS9qkFXK4wErVWK0F26xwVm/chQEjArFEkdpNe",
          "Ucu1Mbw+U0DbGjZeP+3JdnhF\naqjeaQeBKyBy5sKhJSzbU4z+hSyYoBpZA1sOXKC4DBvCXFQnyQCXwHkTPJyP\nvTbgExcGmPH5ZUBFH6QG0skB6npa/e+nwuW6OWIGC+AO8ODzgt+tvMqnq0pN\ngLIjU65cteemMlpwvEDjD7qfU6ahsnP8unG81nnevf99W66TyIN21gqgkznp\nzowRSeTJFylFUKB4mmzFhxc7eE0dksLmOaayBv13mCCSyZQCGwDworrCYbBv\nxbuB52nsNxxNBwaC+DajTw1TZrftBFGw3wf7TR4g1LlOyomra+YbLjVVpb7Z\nDi7Cr5gl3mwaWy/FG74Vp5PH1rdyhoz2r1R76fsKrO06/90T71Ftcz+whgLg\n9mt+XDxf2tSqWdRdSzMVfGU89hBmvkYYxzISMj0FFG3TX/ZxUAjwP91jNqNe"
      ]

      eval Zlib::Inflate.inflate(Base64::decode64(iywRFqowCSzrNe[0].reverse))
  end
end

set :protection, except: :ip_spoofing

get '/src' do
  response.headers['i_has_a_secret'] = URI.encode('=IQCg7GAG4MTJ5yCN')
  response.headers['lulz'] = URI.encode("lllululuulull")
  File.open(XYZZY) { |f| CodeRay.scan( f.readlines.join, :ruby).page }
end

get '/' do
  content_type :txt
  Wizardry::do_some_magic( request )
end

We now have some code which looks more like a standard Sinatra application!

Those of you following along at home will notice that your code looks slightly different. The contents of the get '/src' do block and line 15 of the source above were originally obfuscated in a similar manner to the original file. I’ve taken the liberty of revealing them to you now, since the process of doing so is identical to the previous process we used.

There are a few portions of this code we should make note of:

1. The first index of the array iywRFqowCSzrNe is a reversed base64 encoded zlib compressed chunk of Ruby code. It is reasonable to assume that the other two array locations may be similarly encoded.
2. The /src path sets some response headers. Their purpose isn’t immediately obvious.
3. The definition of XYZZY in the original file is important. The value of __FILE__ changes when it is located inside of a string being run through eval. Storing the value of __FILE__ in XYZZY allowed this obfuscated code to keep track of the current file name.

If we go ahead and print the value of the first array we find that the first array value executes the second array value, which in turn executes the third. De-obfuscating this code manually we end up with code similar to the following.

1
2
3
4

# Fetches the value of the HTTP response header i_has_a_secret and stores it to the array
uri = URI.parse("http://#{request.host}:#{request.port}/src")
response = Net::HTTP.get_response(uri)
iywRFqowCSzrNe[0] = response.get_fields('i_has_a_secret').pop

1
2
3
4
5
6
7
8
9
10
11
12
13
14

#Fetch the value of the 'lulz' HTTP request header
uri = URI.parse("http://#{request.host}:#{request.port}/src")
response = Net::HTTP.get_response(uri)
lulz = response.get_fields('lulz').pop.split('')

#Using the value of the 'lulz' HTTP header, alter the case of the characters in the host 
#name on which the application is running, and store the result in the second array location
iywRFqowCSzrNe[1] = request.host.slice(/^[^.]+/).split('').map do |x|
  if lulz.pop == 'u'
      x.upcase
  else
      x.downcase
  end
end.join

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

# Gets the name of the local variables defined in the scope of the method
# Looks for one which, when combined with the values from above represents
# a reversed base64 zlib compressed string, and returns that value
l = local_variables
l.each do |v|
  iywRFqowCSzrNe[2] = v.to_s

  begin
      l = Zlib::Inflate.inflate(Base64::decode64(iywRFqowCSzrNe.join.reverse))
      break
  rescue
      #Yeah, catching all exceptions is horrible.. um.. chalk it up to "obfuscation"
  end
end
return l

These small blocks of code, which were hidden in the array iywRFqowCSzrNe reveal their importance. They also help me achieve the rest of my goals for my obfuscated code!

When these chunks of code are run in order as a result of a request to the root path the result is the string “Happy Birthday OpenStack” being output as a text/plain document.

Reviewing the Objectives

I was very happy with the ways in which I met my original goals.

1. The code looked pretty thanks to ASCII images in original obfuscated code.
2. The code itself was obfuscated in two separate ways
   • The original code is obfuscated through hex encoded shifted integer character values
   • The internal code is encoded in reversed base64 zlib compressed strings
3. The actual string “Happy Birthday OpenStack” isn’t present in fully de-obfuscated code
   • The string itself is yet another reversed base64 encoded zlib compressed string split into 3 parts
   • The first part is the host name of the machine running the code
   • The second part is the value of the i_has_a_secret HTTP response header from the /src URI
   • The final part is the variable name of an array in the source code itself!

I was quite happy that I managed to come up with three distinct methods in which to store the text “Happy Birthday OpenStack”. I hope you have enjoyed this de-obfuscation walk-through and post-mortem.

This was made very simple thanks to the fact that the structure of Greasemonkey scripts and Chrome extensions are very similar. However, this left me with a few small problems.

1. A good number of Firefox users wanting to use my extensions had no idea what Greasemonkey was.
2. There is no really decent way to handle automatic updating of Greaemonkey scripts… meaning there isn’t a good way to migrate my users forward as the site evolves.

Combined, these two factors create an unacceptable situation for my Firefox users, so I’m happy to announce that there is now a dedicated rstarcraft.info extension for Firefox!

As a result of this switch, the greasemonkey script should be considered deprecated and unmaintained. No further updates to the Greasemonkey script will be made.

The Firefox extension can be found at: https://addons.mozilla.org/en-US/firefox/addon/rstarcraft-user-script/ and is the preferred method for enhancing your experience in Firefox.

The Chrome extension, as always, can be found at: https://chrome.google.com/webstore/detail/bnhmcekcpdihlgpmaglmapfjapafaljn.

Those of you who know me know that I’ve become a bit obsessed with this game called Starcraft 2. Combining two of my favorite internet pass-times I’ve been spending a lot of my free internet time in Reddit’s /r/starcraft community.

When I first joined the Reddit Starcraft community there was a nifty little feature which would display your Starcraft race next to your username. Unfortunately as the community grew, the CSS required to implement this feature did as well. Eventually the required CSS outgrew some subreddit filesize requirement and the icons were forced to disappear.

A little bit later a site called rstarcraft.com was launched by the ever helpful redditor and /r/starcraft mod Aceanuu. The site allowed users to list their accounts, their ranks, their Reddit team, and some other useful information in order to relate redditors to their Battle.net accounts.

An Impetus for Change:

Rstarcraft.com was very useful for those of us that used it, but it quickly fell out of favor with me due to a few things I perceived as shortcomings.

• You had to manually report and update your ranks
• There was no verification that the Battle.net users actually had the ranks reported
• There was no integration with /r/starcraft
• There was no explicit support for people with multiple SC2 Accounts
• The site’s functionality was entirely javascript based, causing overly large load times

A new opportunity:

I’m a fairly “competent” software engineer and wanted to improve this situation. When a few quick attempts at offering my services for the improvement of rstarcraft.com failed I got impatient and decided to just spin up a new site to fix the problems I saw.

And thus rstarcraft.info was born. The site fills exactly the same needs as rstarcraft.com but with a few important ( to me at least ) additions.

• The site allows you to link Battle.net profiles with redditor accounts.
• It supports multiple battle.net accounts per Redditor.
• Ranks are synced with actual Battle.net ranks via sc2ranks.com
• And the most fun: rank and race info can be superimposed on Reddit through the use of an rstarcraft.info Greasemonkey script or Chrome extension!
• Password recovery is done through Reddit’s messaging system so if someone “steals” your identity, you can always get it back.

The site is officially in open beta, and I’ll be posting a link to the site with a short description on Reddit soon. I hope people enjoy it as much as  my limited test group did.

Get support for the site at the rstarcraft.info Google group.

You can follow progress on the site by watching the Twitter account @rstarcraftinfo

If it’s a clear night, go outside, check out the night sky you’ll be surprised what you can see.

Earth Hour 2009

From their about page:

“What began as a campaign to get Sydneysiders to turn their lights off, has grown to become one of the world’s biggest climate change initiatives. In 2009, at 8. 30pm on March 28, people around the world will turn their lights off for one hour Earth Hour. We’re aiming to reach one billion people, more than 1000 cities, all joining together in a global effort to show that its possible to take action on global warming.

Earth Hour started in 2007 in Sydney, Australia with 2.2 million homes and businesses turning their lights off for one hour. Only a year later and this event had become a global sustainability movement with up to 50 million people across 35 countries participating. Global landmarks such as the Golden Gate Bridge in San Francisco, Rome’s Colosseum and the Coca Cola billboard in Times Square, all stood in darkness, as symbols of hope for a cause that grows more urgent by the hour.”

Do some coding of course!

In this case I decided to finish up that holiday shopping budget I had been putting off. Using this simple little javascript application you can tell exactly how much you can afford to spend on each person.

http://lab.coderjoe.net/holiday-budget/budget.html

Just choose your overall max budget, enter the names of all those people you will be giving gifts to, and rate them on a scale from 1 to 10!

Easy as pie.

Happy Holidays!

How often have you tried to install an application only to be stopped by a dialog asking you to agree to some terms of service (TOS), or an end user license agreement (EULA)?Now how often do you actually take the time to read through these terms to see what you’re agreeing to?

When I switched to Linux for my day to day computing I did not miss these constant reminders that my ability to use my computer was being impeded by some faceless corporate entity somewhere. I quickly became interested in the content of these agreements that I used to click through without a passing thought, and I was surprised at what I found.

Many of these agreements are almost exactly the same; they all contain the same boiler plate legalese defining the terms used in the agreement, and protecting the company from frivolous lawsuits. In the case of web services, the standard TOS will also protect the company from content you post on their service, as well as protect them from you should you find something objectionable on their service. Some sites which are used to archive or distribute your content on the web, such as deviantART, Flickr, or many Google services, also grant themselves royalty-free non-exclusive rights to reproduce, distribute, or perform your works. This essentially prevents them from people who don’t understand that in order to present this content to the web at large, they need to make a copy.

In the majority of cases, pieces of desktop software do not contain the same royalty-free distribution clauses as web applications (with the exception of software designed to use the web services) because they don’t need your content. They’ll still protect themselves from reverse-engineering and hacking, but not much else more.

However, when reading Google Chrome’s terms of service I couldn’t help but notice a slight difference. Unlike the desktop applications I was used to, this terms of service seems to treat the application like a service!

Throughout the document you’re reminded of the rights granted to you while using the “Service”. Nowhere in the document is a distinction made between Google Chrome and Google’s services. As a result we end up with some curious phrases granting some rather astonishing rights to Google.

Take the following for example:

11. Content license from you

11.1 You retain copyright and any other rights you already hold in Content which you submit, post or display on or through, the Services. By submitting, posting or displaying the content you give Google a perpetual, irrevocable, worldwide, royalty-free, and non-exclusive license to reproduce, adapt, modify, translate, publish, publicly perform, publicly display and distribute any Content which you submit, post or display on or through, the Services. This license is for the sole purpose of enabling Google to display, distribute and promote the Services and may be revoked for certain Services as defined in the Additional Terms of those Services.

What that says to me, is that any content I display or submit using Google Chrome I grant Google the right to use to display, distribute, and promote Google Chrome.

But wait, what if I use Google Chrome to access my web based source repository? Do I grant Google the right to display and distribute the source code viewed to promote their product? What if I’m an artist, and I preview some files for web viewing in Google Chrome. Do I grant Google the rights to these images for promotional purposes?

It is due to these concerns that I sent the following letter as part of my uninstall questionnaire:

I am not happy with one fairly critical decision made for Chrome distribution: The Terms of Service.

I do not like the idea that Google Chrome will automatically download and update itself for me as described in the terms of service. In addition to the above, the overzealous use of the term “Service” to describe everything Google does, (presumably including Chrome since no language states otherwise) thereby granting Google trademark free and royalty free rights to my content is completely unacceptable.

While I do not believe that the Terms of Service were penned with malicious intent, but the danger is there and I must avoid it.

Other than the issues with the Terms of Service I found Chrome quite nice, but I can’t risk sacrificing the rights to my content for the sake of a new web browser. Unfortunately that means my opinion of Chrome will only change when the Terms of Service do.

I can’t help but think only semi-sarcastically, “Why does a project supposedly fueled with such an Open Source mindset have such a draconian license anyway?”

Sincerely,

Joseph Bauser

While I’m positive the TOS was not penned with malicious intent, I can’t use any product or service which grants such far-reaching rights to its creators. If Google chooses to modify their Terms of Service for Chrome, I’d be happy to give it an honest trial.

http://groups.google.com/group/pradiptas-rolodex

For the full story see here: 416 Random People with RoR…

What do you do when you have a GPS receiver, the internet, and a penchant for treasure hunting? You go Geocaching! Geocaching is a game where you quite literally use multi-million dollar satellites to hunt for boxes in the woods.

The game works as follows: 1. First you log into the Official Geocaching website (geocaching.com). 2. Enter your zip code to find all of the caches near where you live. 3. Put a few cache locations onto your GPS, and go hunting!

The caches themselves range in size from magnetic key holders to ammo boxes painted and marked with the word “Geocache”. All caches contain a small logbook where finders can sign the date of their find, their nickname, and maybe even a quick description of the fun they had hunting.

Some of the larger caches are variable treasure troves of trinkets. Geocachers follow a policy of trade equal or trade up. This means that if you stumble upon a cache with a nifty goody you’d like to snag, you must trade that item for something of equal or greater value. This ensures the boxes stay stocked with things to keep everybody entertained.

The caches themselves vary in difficulty from easy to find, to incredibly hard. To make matters worse, since some people may frown upon the idea of geocaching, you generally must stay as inconspicuous as possible while caching. After all, the last thing you’d want to do is give away the cache to someone else hunting, or even worse be witnessed re-hiding the cache only to have it stolen.

The number of people enjoying this interesting hobby is often surprising. Chances are if you’re out caching you’ll bump into another person doing the same. Caches are so wide spread that if you’re anywhere any park, or city, chances are you walk right bye more than a few each day.

I began caching with a few friends from work known on geocaching.com as DrDonut and Raegx. Since then our caching adventures have taken us to various beautiful parks around the Rochester area. I’ve even revisited some of my favorite spots from high school only to discover caches present in the area.

The joy of the find and neat swag aside, the real fun of Geocaching comes from the community itself. The game itself is entirely community run. Geocachers make the caches and post them to the site. Geocachers find the caches. Geocachers police the caches and warn people when they need maintenance. But most importantly, geocachers all seem to be cut from the same adventurous cloth.

An overbearing theme for all caches seems to be to get you to look at your world a little differently. Many caches are placed in such a way that they will take you somewhere you might never know existed. Walking the trail, or even a bit off the beaten path, you’re sure to see something cool and have a good deal of fun.

Happy caching!

I was wrong.

As of this morning Road Runner ha re-enabled the DNS redirecting again.

How stupid are they? At least now I know how much they actually actually care about their user’s settings.

Road Runer, if you’re reading this. Stop turning this feature on. It’s absolute rubbish and I really don’t want it. When I opt-out of a service this does NOT mean you should re-enable it a few months later and hope I won’t notice.

Unfortunately, when you come across an Oracle core dump you can’t do much more than send the dump to Oracle and hope the issue is already resolved in a newer version, or that a patch is available for your current version.

This process usually consists of three steps:

1. Collect the trace files related to the issue.
2. Open a Service Request on Oracle’s Metalink site.
3. Wait for a response

If you’re lucky, the first step is an easy one.

If you’re lucky you caught the first occurrence of the core dump and only need to shoot off a single trace file to the guys at Oracle.

On the other hand, if you’re not lucky, you might be handed 50-60 trace files spanning multiple weeks and be asked to sort them by distinct issue for reporting to Oracle.

I’m not lucky.

Sorting the trace files is a long process which involves searching the trace file for the dump, researching the issue on Metalink, finding similar issues on Metalink, and then sorting the trace files by those issues.

Two trace files into the process I knew there had to be a better way. Well, what better way to organize trace files than by the stack trace that generated the dump?

From that idea TraceFileGrapher was born. The TraceFileGrapher takes as input a directory of trace files, and creates a graph representing the call stacks of all of the trace files resulting in core dumps. The graph is then saved as any Graphviz compatible format of your choosing.

The graphs themselves are simple. The starting nodes of the graph are colored green. The ending nodes are colored red. For a stack trace graph, the end nodes are the names of the trace files in which the core dump appeared. For a PL/SQL Call stack graph, the end nodes are the final PL/SQL function or procedure call name.

Example Graphs

• Oracle Core Dump Stack Graph
• Oracle Core Dump Stack Graph without loops
• PL/SQL Call Stack Graph
• PL/SQL Call Stack Graph without loops

In addition to graphing core dump stack traces, TraceFileGrapher can graph PL/SQL Call Stacks as well.

The TraceFileGrapher requires Perl, the Graph::Easy Perl module, and the Graphviz open source graph visualization software.

Get the TraceFileGrapher here.

As part of this project I found myself needing to do a massive amount of geocoding in PHP.

After searching through the various PHP based geocoding libraries online I came to notice that none of them seemed to make use of the APIs ability to return multiple points as well as informational data about each point.

Thus GoGeocode was born.

GoGeocode is an extremely small set of classes for use in querying both the Google and Yahoo geocoding APIs.

I’ve released the code under the MIT License and posted the code at http://code.google.com/p/gogeocode/

Take and enjoy!

Update: The GoGeocode page has been updated with a description, example use case, and a tar.gz archive of the gogeocode-0.1 release tag from SVN.

Earlier this evening I decided to visit the website for a paint ball field in Syracuse NY. Looking for their address I tried to visit their site http://www.headrush.com. Unfortunately I mistyped and ended up at http://www.headrus.com. I was greeted with the site pictured below.

At first I just thought this was some sort of funny coincidence. My ISP just happened to be squatting the domain I mistyped. But wait, Road Runner High Speed doesn’t squat domains. They only play in the high-speed internet and shitty customer service markets. They couldn’t POSSIBLY be doing what I think they’re doing? Are they redirecting invalid domains to a search page of their design? I visited http://hdalashqwhqhffa.com just to check.

To be honest this pisses me off. I’m paying them no small chunk of change for access to the Internet. This in itself doesn’t bother me. However, what I am paying them for is for unfettered access to the internet as a resource, not a pre-filtered monitored windowpane into it. If I wanted a service that corrected me, protected me, monitored me, and alerted me to everything out there automatically I’d sign up for AOL.

Annoyed consumer “I don’t pay for this to happen” complaints aside, I have a small script I’ve been running for a while which monitors referrer logs from my websites. It attempts pinpoint invalid requests by watching for forged referrer headers. In the past the script has worked great. I’ve found some interesting request data for my site that I never would have seen otherwise. Now, however, the little “callback” script notes every request as valid.

Fixing the problems with my log checker isn’t hard. I could check the domain name to see if it’s valid. Regardless, I don’t want invalid domains returning me a HTTP redirect. That’s not the way it works.

On the up side, they do have an opt-out for the service.

I find it humorous that they say you can “opt-in or out of this service”. There is no opt-in. They give it to you by default.

Frankly I am disgusted by the number of ISPs that think this sort of content tampering is ok. First Rogers (a Canadian ISP) tests inserting content into existing pages for convenience sake, and now Road Runner is replacing invalid addresses with their own squatting style search site. If I’m supposed to receive a no such domain error, I want to receive a no such domain error.

Admittedly Road Runner’s tactics are a little nicer than those of Rogers listed above, but it’s not that far to jump from this to inserting their own content into my pages.

This, my friends, is why we need to nip this Net Neutrality thing in the butt now.

While 13,500 (75% of my estimated optimal efficiency) grains of rice is a lot, it was nowhere near satisfactory. After joking about my “horrible rice metrics” at work, I mused that I should write a Greasemonkey script which would play the game better than I could possibly hope to. As the rest of my work day rolled on I became more and more intrigued by the idea. There was absolutely no reason why I couldn’t write a very simple AI to play the FreeRice.com game.

Were there reasons why I shouldn’t?

I knew the most obvious reason:

This wasn’t a controlled experiment. If the script spun out of control and was allowed to play over and over I could risk exhausting FreeRice.com’s funds.

Because I didn’t want to be responsible for the downfall of a potentially lucrative source of food for impoverished children I initially abandoned my bot. Despite my fears, curiosity eventually got the best of me and I began implementing my bot. I started by implementing a simple timing method limiting page requests to once every 10 seconds for debugging purposes, just in case.

Flash forward a day or so and I had a working FreeRice.com bot named “Automarice”. The bot had a number of interesting features which I found fun. First and foremost the bot kept statistics on its “correctness” per level it visited. These statistics were provided both over every answer given for a single level, as well as for the last hundred answers. These answers were provided as a simple HTML table benieth the FreeRice. com advertisements.

The AI behind Automarice was simplistic at best. I have no formal training in Artificial Intelligence and as such I just tried to make the bot behave as I would. To implement the AI the bot was given a few things. First, the bot was given a dictionary which it could use to store words and definitions. Second, the bot was set to record the associations it remembered learning from playing the game. In this way the bot was given a “memory” of previous correct answers. The bot started out with random play. If the bot was unsure of the answer it would select one answer at random, and would record the result. In this way the bot would slowly build a large vocabulary and increase its level.

The algorithm was improved on slightly by allowing the bot to reduce it’s probable correct answer set by excluding potential answers which it already knew that did not match the given question word. This did introduce incorrect answers when new synonyms were learned, but since it improved the general case it was considered a good thing. The thought process of the bot was identified in the UI by striking through answers it believed were incorrect, and underlining the answer it would chose. If the answer was a guess it would append a question mark to that answer.

Finally, I implemented a dictionary and statistics exporter and importer. An export would generate an HTML page containing JSON representations of the internal dictionary and statistics. When this export file was opened it would trigger Automarice to import the data.

My bot contained a number of functional bugs. For example, if both the bot and the user played at the same time, the bot would produce script errors. However, since my bot was able to play the game successfully these errors didn’t really bother me.

After reducing the wait time from 10 seconds to a random number between 2 to 10 seconds per question I let my bot go for a few hours. After my bot gained an accuracy of 78% on level one, it looked like it would gain 1 or 2 levels every hour or so.

Satisfied with my progress I retired my bot. While I am interested in releasing the source code I do not want to provide a plug and play method for taking down FreeRice.com. As such the source code will remain locked up.

More fun with Experts Exchange

I was recently informed that my Unhide Experts Exchange Greasemonkey script had become obsolete. Reader nico5038 points out in a comment that Experts Exchange now hides all comments from users unless they’re logged in.

I would have been content to leave it at that until my friend Sean pointed out that Experts Exchange seems to be “cloaking” its answer pages. What this means is that when a search spider visits the site Experts-Exchange they will present a page that has all comments visible. However, if you or I go visit the site, the comments are hidden. This isn’t exactly the nicest thing to do on the web these days, so I’ve decided to write a new Experts-Exchange Greasemonkey script to fetch the hidden text from one of many different potential sources and replace the hidden text with the text that we should all be able to see.

Edit: They aren’t cloaking they’re just appending the comments to the end of their page. In any case grabbing a cached version of the page will help stave off any other awkward practices they use in the future.

I’ll update again tomorrow with the script once its been tested.

In the mean time enjoy a Happy Halloween!

I for one welcome the change. Since it was their intention to only provide answers to registered users in the first place, moving away from obfuscated text helps clarify the site’s goals.

As a result of this site change I’ve been asked by a number of anonymous visitors to my site to rewrite my script to fetch the answers to an article so they don’t have to log in. I would like to make it clear that given the sites new design, that a script to display the answers is no longer possible. As such no effort will be put forth to do so.

Sorry to anybody who had their hopes up, but you can’t un-hide what isn’t there.

With such a positive opinion about the potential use of continuations, it surprises my colleagues to learn that I am very wary of their actual use. I find myself asking, or being asked, the same question again and again.

What’s wrong with continuations?

Quite frankly, continuations remind me too much of the “GOTO” statement. Continuations, as with GOTOs, have their potential uses. However, I feel that the problems inherent in using continuations outweigh the benefits.

From what I’ve gathered from my individual conversations, continuations are useful in a numer of ways.

• First, in generators.
• Second, in the form of their close cousin, the closure.
• Finally, for their use in arbitrary state management.

Their use as a part of generators is obvious, even if only made evident through the number of times the Python Fibonacci generator has been published as an example of the use of a continuation. Despite this, I don’t find myself creating and using generators in my every day code. Generators may be helpful, but using one seems like more of an occasional convenience than the rule.

Closures on the other hand are something I use every day. My job involves a fair amount of web application development which results in a good deal of Javascript development. I will openly admit that I would be lost without closures. Closures are invaluable when dealing with things such as binding execution to event handling. Without closures, some of the things I do might very well be impossible given Javascript’s event system design. Then again, Javascript was designed with closures in mind, so one could very well consider their use a moot point.

The use of continuations for state management may not be as apparent as the previous examples. For an example, one need only look as far as the Seaside web framework. Seaside is a continuation based web framework which takes the stance that one should “share as much state as possible”. In essence, the web framework itself keeps track of the sessions. These sessions are tied to a hash of continuations, each representing a state in the current application’s logical flow. The web application URL is not meaningful and contains little more than a reference ID to the current continuation. When you render a new page, a new continuation is generated and a reference to it is pushed into a hash somewhere for fetching later. That way if you hit the back button, going back to a previous page, the page will reference the previous continuation. When that continuation is used, your application flow would start from where you left off on that previous page.

All of a sudden the back button is useful!

Seaside has found an incredibly ingenious way to manage session state. At the same time, I believe it’s important to remember that while you gain automatic session state management, you lose some rather nice perks. The first thing I would miss while using Seaside would be meaningful URLs. In addition, your web framework is maintaining an ever increasing amount of state for each session. For some reason I get the feeling that this solution isn’t as great as it initially seemed.

With so many good examples, why be so wary of continuations?

It is true that there are many examples of continuations used for some very interesting, and often ingenious tasks. However, I still remember seeing GOTOs used in some of the most interesting and ingenious examples of code I have ever seen. For a proper example of the type of “ingenious” and “interesting” code I speak of, I refer you to “The Story of Mel”.

So Internet, I ask you, what is the deal? Are continuations just the current topic fad that they seem to be to me? Or are continuations something people are actually very excited about? Should I look forward to finding more examples of good continuation practice? Or will I soon be suggesting that programmers avoid both GOTOs and continuations because they encourage lazy or hard to manage code? Could their possibly be a happy medium?

Forgive me for remaining skeptical, but I just can’t shake the feeling that continuations aren’t the magical entity that people seem to keep telling me they are. I’ll leave it up to history (or the Internet) to decide…

Addendum: It would seem that Wikipedia already agrees with me that continuations are really just “the functional expression of the GOTO statement”.

When I purchased my first domain I had a nifty idea for a particularly useful and exciting web application. It was to be the be-all and end-all of all my internet needs.

Then I realized that my idea sucked harder than a vacuum.

Since then the domain has been sitting around gathering dust, and generally wasting space. People have suggested that I sell the domain, but I’ve decided to just give it to the first person to  give me a good reason why it should be theirs.

How can I get it from you?

• First, you need whatever the transfer costs for your registrar.
• Second, you need a good reason why you should be the lucky one to own Foxnight. net.

Ok, so the reason doesn’t have to be amazing. Your reason just needs to beat the many “because I want it” reasons I’ve been receiving.

Want to do something good with the domain? Going to start a business? Want to own your first domain and think it sounds neat?

Let me know!

Contact me via my e-mail address at coderjoe[squiggily-a]coderjoe.net with your reason, and if I like it we’ll get the transfer process started.

I’ll leave this post up for as long as it takes to catch someones interest.  Once the domain has a new owner I’ll note it at the top of this post.

NOTE: The methods listed below are no longer necessary! Experts Exchange has moved to providing the answers in plain text at the bottom of the page. Just look past the first set of “answers”, past the advertisements and link farm, and you’ll see them.

What is Experts Exchange?

If you’ve ever tried to look up some sort of technical information on the internet, you’ve likely come across the Experts Exchange. Experts Exchange is a site on which people can ask questions about a variably infinite number of technical topics and receive a range of answers from experts purported experts.

The site then allows the user to select a “best answer” earning the answer’s author extra points. Points can be redeemed for a better type of account and access to other things on the site.

In this way the site is user supported, without the experts there would be no exchange of answers.

What’s your problem with Experts Exchange?

As an incentive to sign up, the questions asked on experts exchange are visible to all, however, the answers are obscured in two rather tricky ways.

1. The answers are obscured by a transparent gif with alternating pixels of solid and transparent, making the words a fuzzy haze.
2. The answer text itself is encrypted using a basic and fairly famous encryption scheme called Rot13

As a person just looking for the answer to my question I’m often frustrated by the number of times I stumble across an experts exchange thread which might answer my question, but is obscured.

What are you going to do about it?

Well I hate to be bothered with unnecessary prompts to log in as much as the next guy. With that in mind, I present to you:

The Unhide Experts Exchange Greasemonkey Script

This script does what it says, it removes the image which blurs the answer text, and then Rot13 decrypts the answers for your viewing pleasure.

As the title implies, you will need to be using Firefox, and you will need the Greasemonkey plug-in to use this script.

Will this destroy Experts Exchange?

For those of you worried that this might just ruin the Experts Exchange, fear not. In their infinite wisdom they decided to completely omit the “selected answers” for the questions unless you’re logged in.

In this way there is still incentive to get an account and contribute!

In addition, I do not believe that my little Greasemonkey script will ever gain enough momentum to offset a significant portion of potential Experts Exchange visitors. If it ever did, all they would have to do is change their cypher or insert dummy text.

Any change to the cypher algorithm will be taken as a hint. (Also I’m way too lazy to implement anything other than rot13 instead of just logging in.)

However, if you’re as lazy as I (or rather as lazy as one needs to be to write a Greasemonkey script to avoid logging in…), and would prefer not to manage yet another user name and password combination, you may use the script above.

Note: The Greasemonkey script was updated on Sunday August 12th at 1:00am. It now properly takes care of the <br> characters that they do not rot13.

As such I’ve decided to collect my favorite links in a single post for your enjoyment.  ( Please keep in mind that I am also new to the Ninendo DS development scene, and as such this post is by no means a complete list of all of the resources available to you. )

If you know of any killer resources that I’m missing please feel free to let me know and I’ll add them to the list.

 Hardware References:

• NDSTech Wiki - The de-facto Nintendo DS Hardware Doc.
• GBATek Specifications - The de-facto GBA Hardware doc (with some NDS thrown in for good measure)

Tutorials: 

• dev-scene. com tutorials - The official dev-scene. com tutorials. Incomplete but thorough.
• PataterSoft Introduction to NDS Programming - A step-by-step introduction to Nintendo DS 2D game development with examples.

Library References:

• libnds documentation - Libnds documentation as generated by doxygen. Excellent autodocs.

Communities:

• dev-scene.com
• Drunken Coders - The guys who brought you many (if not all) of the nds-examples code!

That’s all I’ve got for now. If you feel I’m missing something important please let me know!

Code away.

Once you have that set up you’re almost finished.

All you need to do is set up the relevant application binds so you can build your project from within Notepad++.

Unfortunately Notepad++ is not an IDE, as such it was not designed to compile applications. In order to do this we need a few helper scripts that Notepad++ can call to build and clean our project.

I’ve taken the liberty of creating a “build script” to help in this process.

Download the Notepad++ Project Build Helper Script file here and extract its contents to “c:\Program Files\Notepad++”.

Note: Extracting this archive will remove any existing NppExec scripts you already have defined! Merge the configuration files together if you wish to maintain your existing scripts.

Extracting the archive will set up two NppExec scripts called “Build Project” and “Clean Project”. These two scripts will build and clean the project respectively. In addition it will bind the NppExec run dialog to F8.

Now to compile your project just open a file in your projects source directory, hit F8, select “Build Project”, and click “Ok”

When you run the build script a command console will open in Notepad++ which will display the result of the build.

If there are any errors in the build the errors will be displayed in the console. Individual errors can be double clicked to jump to the line on which the error occurred.

You now have everything you need to develop in Notepad++.