boltzmann in action selection

I've been replicating across some of the functionality of my TD(lambda) bridge player project into Java to test in Tyrrell's world. Last night, I fixed a kind of obvious bug that had slipped me for a week or so and made it work hopefully properly, at least for the TD(0) case.

The performance in Tyrrell's world wasn't all that great though, so I started decreasing the softmax temperature parameter to make it a bit more greedy... here's an interesting result:

with temperature ~=0.8:
Steps: 255 Sexed: 0
Steps: 11 Sexed: 0
Steps: 263 Sexed: 0
Steps: 274 Sexed: 0
Steps: 211 Sexed: 0
Steps: 257 Sexed: 0
Steps: 253 Sexed: 0
Steps: 251 Sexed: 0
Steps: 254 Sexed: 0
Steps: 307 Sexed: 0
Steps: 428 Sexed: 0

with temperature=0.14:
Steps: 391 Sexed: 0
Steps: 392 Sexed: 0
Steps: 393 Sexed: 0
Steps: 393 Sexed: 0
Steps: 392 Sexed: 0
Steps: 391 Sexed: 0
Steps: 394 Sexed: 0
Steps: 391 Sexed: 0
Steps: 394 Sexed: 0
Steps: 391 Sexed: 0
Steps: 392 Sexed: 0
Steps: 393 Sexed: 0

I'd have thought survivability was random if you did practically nothing, but I guess not? Anyway, that's obviously a bit too low.

MiniJavaBackend

Well, that was one hoor of an assignment. We were basically given a month's extension - I had it practically finished except for the extra CALL tile handling - function prologue (callee-save and copying formals) and epilogue (restoring callee-saves and setting up return value) when it was originally due, and left it on hold until a few days ago.

Actually getting CALL working even half-properly was a total headwrecker. The documentation was shit, and there are still unexplained issues that I don't understand - for one, incoming parameters get copied into temporaries that are never referenced... instead, what look like older temporaries are used. Sob.

czt -> community Z tools

Here's a tip for anyone who needs to put a Z specification together quickly.

Never, ever, ever, ever, EVER use CZT. It is the most frustrating, dog-slow piece of shit software package I have used in my life, without exaggeration. With jEdit it's just unstable and crashy enough to warn the user away immediately.

As a plugin for Eclipse, it starts out nice, then after about one page of simple Z schemas, it parses the buffer repeatedly on every single keystroke, even if you tell it not to in the configuration page. It throws syntax errors on things that look right according to any examples of Z I have ((i,j) of a natural x natural type referenced in an axiomatic definition).

It started out in a promising fashion, and the output through latex looks nice, but now I'm in a position where I don't understand the errors it's giving me (after the unbelievable 3 minute delay every time I make a change (ie: hit some keys)) so I can't even convert what I have to tex and work on that directly - I have to delete all of the code that causes errors as quickly as possible so it will parse the buffer successfully.

Never, ever touch this stinking pile of pain. It's just not worth it.

reinforcement learning with function approximation

This is much harder than I thought. I've got a series of tests (not very modular at all - they're split into only two source files and cover test-first unit tests and empirical performance measures) which examine the components of the system in some way... the neural network seems to be functioning reasonably, although it's less accurate than I'd expected for pure numeric output, unless I format the output as a binary string again... annoying.

Not sure now whether to have the TD net represent its estimated values like this or keep it as an expanded sigmoid for now.

Here's its performance just now on a 30-room random walk test - there are two actions: 0 is left and 1 is right. All rewards are 0 except when entering the terminal state (furthest room to the right) which gives a reward of +100. I kind of expected much better performance ages ago, but I guess it's harder to get these systems right than I thought. Although it completes the task in 15 moves a few times (the minimum amount), it often goes wrong.

learningRate: 0.5
steps taken on run 0: 697556
action 0 chosen: 91.2195%, action 1 chosen 8.78051%
steps taken on run 1: 67
action 0 chosen: 38.806%, action 1 chosen 61.194%
steps taken on run 2: 99
action 0 chosen: 42.4242%, action 1 chosen 57.5758%
steps taken on run 3: 121
action 0 chosen: 43.8017%, action 1 chosen 56.1983%
steps taken on run 4: 115
action 0 chosen: 43.4783%, action 1 chosen 56.5217%
steps taken on run 5: 27
action 0 chosen: 22.2222%, action 1 chosen 77.7778%
steps taken on run 6: 15
action 0 chosen: 0%, action 1 chosen 100%
steps taken on run 7: 35
action 0 chosen: 28.5714%, action 1 chosen 71.4286%
steps taken on run 8: 19
action 0 chosen: 10.5263%, action 1 chosen 89.4737%
steps taken on run 9: 17
action 0 chosen: 5.88235%, action 1 chosen 94.1176%
steps taken on run 10: 17
action 0 chosen: 5.88235%, action 1 chosen 94.1176%
steps taken on run 11: 15
action 0 chosen: 0%, action 1 chosen 100%
steps taken on run 12: 15
action 0 chosen: 0%, action 1 chosen 100%

Sometimes it gets stuck in some weird state (local optima?) and never really learns anything. At this point, the action selection policy is just the action with the best estimated value unless the two actions appear worth the same, in which case it's a 50/50 tossup, or the epsilon-greedy function chooses a random action with probability 0.08.

I'm thinking the next step is to implement a softmax action selection method and maybe another test.. or, I dunno, actually carry on with the functional requirements of the thing. Time runneth short.

Bin Ankh

This was on the pavement on Gardiner Street, near my apartment in stabsville, Dublin 1:


I wonder how it came to be...
And while I'm at it, here's a nice one from DCU. Nice!

Japan, America and pacifism

Tangentially, tacked on to this post about consumer robotics in Japan is a very interesting discussion on the degree of pacifism in Japan since WWII, movements (or lack thereof) of American politics over the last 50 years and wherever the rambling trail leads. Read it!

software for truly lame people: WhiteSmoke

I got an ad from FreshDevices in my email today, and instead of deleting it like 99.9% of the other ones, I had a look and saw mention of a program called WhiteSmoke.

The ad text read:

Write like an English pro using this software. With full-text analysis solution, convert simple English sentences to become more sophisticated automatically.

This premise raised my ire pre-emptively, since if the "simple" English sentences communicate the correct meaning, surely making them artificially "more sophisticated" is either distorting the meaning or at best, fluffing out concise text with superfluous padding. In a sense, de-optimising the text.

So I had a look at the site and found pretty much what I expected.

Before:
He was very interested in our product line, however, he has some issues with our priceing.

After:
He was very interested in our innovative product line, however, he has some significant issues with our pricing.

How does it know whether the product line is innovative, and is it even relevant? Obviously the assumption is that you would always refer to your products as "innovative" whether it is or not, but this just makes the word optional and eliminatable (okay, that's probably not a word, but it should be...).

Ignoring the fact that it corrects the obvious typo in "pricing" (which should be done by another single-purpose tool anyway) it also adds the modifier "significant". This is all clearly in the name of making simple text sound more formal and "clever", but I have two issues with this:

a) People should be able to do it themselves without changing the meaning inappropriately
b) This clearly changes the meaning inappropriately.

If someone doesn't speak enough English to write in an academic tone, they can get by with what they have and use a dictionary/thesaurus to get the meaning across. And a native English speaker has absolutely no excuse to even consider using this thing. Using an automated tool to make your writing look "clever" is the very opposite of clever. It's akin to having a voice filter on your phone that transforms your speech into a different accent (Cockney Skank to marbles-in-the-mouth Rupert). The whole concept is utterly lame and anyone caught buying such a tool should be whipped. Whipped!

TD(lambda) and bridge

I've been doing another all-nighter working on my final year project - a neural-network based TD(λ) reinforcement learning bridge player... I say that, but really I've been hung up on implementing the bridge game model. It's as complex as I worried it would be - that is to say - fairly difficult. And I can't just go to sleep (unless it's for 2 hours... hmm....) as I'm due to demo something at 11:30am - 7 and a half hours from now. Eeehhhh, right. 真不好.... all I have so far is some confusing code and a limited-scope set of testcases. Sleep is sounding better and better!

dodge this....

Apparently NASA is researching the effects of high-speed collisions of tiny meteorites on the moon's surface. The concept is pretty disturbing, since some of the rocks are travelling at up to 35 kilometres per second when they hit the ground. At those speeds, a spacesuit isn't going to stop you from being turned into a smear instantly... so how are we going to survive in space?

As NASA explains it, you might dodge a bullet but still get caught by its shrapnel. Among the questions they want to answer: Are you more likely to get cut off at the ankles by debris spattered along the horizon, or hit from above by material on high, ballistic trajectories?

consequences of blind faith

BBC: Indonesians jailed for beheadings

This really makes me wonder how people can possibly be that vicious and remorseless. This guy and his two friends beheaded 3 schoolgirls because they were students at a Christian school, as revenge (or something) against the Christian community for some previous violence against Muslims.

How could their made-up rubbish (religion) be more important than the lives of innocent civilians? He said at the sentencing that he didn't care about going to jail either, demonstrating a total absence of remorse.

As far as I see it, religion offers only the "blind faith" needed to murder innocents in the name of God and brotherhood. This is the same kind of conditioned psychotic mentality exhibited when fanatical Muslims burned down embassies and basically violently wrecked the place over those Muhammad cartoons. Idiots.

The old Christian phrase "love of money is the root of all evil" is inaccurate. I think the correct one should be "blind faith in religion is the root of all evil", because it allows apparently normal people to dehumanise and then horrendously butcher and murder other civilians.

China's new property law

A quote from a Chinese net user on the new property law passed in this year's "National People's Congress", which turned up on the BBC's news site.

"With the property law, finally, I won't be afraid of losing my bicycle. If I lose it, if it is still in China, it still belongs to me according to the law."

Sarcastic bastard. I love it!

scim + firefox 2

Firefox 2 and 1.5 as distributed by Mozilla are linked against a different version of libstdc++ than the binary distributions of the SCIM input method. I think. Anyway, whatever way it's done, Firefox crashes immediately on startup - running it from the command line produces this output:

/usr/lib/firefox/run-mozilla.sh: line 131: 2198 Segmentation fault "$prog" ${1+"$@"}

I encountered this problem before and worked around it by specifying xim as the GTK input method module... but this meant I couldn't use any input methods.

So it happened again today when Firefox updated itself to 2.0.0.2, and I happened upon a much nicer workaround:

sudo vim `which firefox`

then add:

GTK_IM_MODULE=scim-bridge

a line before "moz_pis_startstop_scripts()".

This allows me to use SCIM in Firefox 2. Happy days...!

careful now... DOH!

Hahahaha.

...

Anyway, I did pretty shit in my fourth year semester one exams - worse than any other semester since I started the course in 2003. I think my average is about 67%, but I can't be sure until I get my Cryptography assignment marked... looks like I only scored 67% on that paper as well, though. Unnnh...

It's weird - I walked out of most of the exams thinking "I nailed that!", but nope. This means I need an average of 75% or better on my three subjects and final year project to get a first class honours degree. Usually I'd say "no problem, I keel you. Badges? We don't need no steenking badges!" but after last semester I dunno what to think - seems like my level of understanding sense was pretty inaccurate...

post-exam stress syndrome

After two weeks of exams and a couple of 4 hour sleep nights in a row, I guess it's natural to be absolutely exhausted. Still, I think all of them went okay except the Graphics fiasco, which was a kick in the face for a lot of us.

Still, as long as I passed it, I can hopefully make up for it with the other stuff... I keep telling myself how I'll work more efficiently in semester two. I'd better...

In other news, this is great reading. In fact all the articles are cleverly written (and where does he find the stuff..??)

It is also important to remember that Glen and Sophie are characters in two separate productions. Hey, it's the magic of post production. Glen then sits down to talk to another new character about Sophie. This is where the seams are really starting to show, as Glen obviously is in a barracks in the middle of the jungle, while the man he is talking to is in an office, in different lighting, and in a different movie. This guy then hires someone to rescue Sophie.

Brilliant.

drivers

Think Dublin drivers are shit? Try this guy...

私は茶を飲むのを好む。

Automatic translation tools are very useful, but it seems clear that there's still a lot of improvement that could be done in the area.

For example... go to google translate.

Enter "Let's drink tea.", select English to Korean and translate.
Paste the result ("차를 마시자.") in to the box and translate back to English.

Surprisingly, the outcome is "Drink the car." What? Right...
It might be a good idea not to be organising any tea parties in Korean using translation tools to communicate for the time being...
Japanese isn't so accurate either: translating "I like to drink tea" to Japanese and back to English yields "I like the fact that brown is drunk". Firstly, that stuff should be solid or you should be seeing a doctor, and anyway, I don't want anybody drinking my brown. That goes straight in the jar under my bed.


Update 4th February 2011

• Now translating "Let's drink tea." to Korean and back results in "Janhaja tea". Eh.


• The Japanese round-trip for "I like to drink tea" is now accurate.

It's a pity I didn't save some more bad examples of which I compiled quite a few, to see how the state of the art (assuming Google Translate is reasonably representative of this) has advanced in the last four years.

exams...

exams exams exams. We got the easy one first, so I spent about 3 days reading and making notes for it and was pretty well-prepared. Now I have just over one day to prepare for the graphics exam which is almost certain to be much more difficult.

Firstly, our software patterns & metrics exam is clearly structured - past papers online for the last few years are all "answer 3 out of 5 questions, each question on a well-defined area of the course: design patterns, architectural patterns, metrics, estimation and testing". That's the whole course. Easy peasy.

With the graphics course on the other hand, we'll be asked to answer 3 out of 4 questions from about 10 different areas... which means we have to study everything. Lame!

athbhliain faoi mhaise!

Happy new year! I did basically nothing on mine (I felt so lame that I went out for milk at 00:10 to wish a security guard and cashier a happy new year).

Thrilling stuff...

Roujin Z?

I think the caption "Robots are looked on as a solution to Japan's ageing population" under the first photo in this BBC article is a bit misleading.

1.

(a) The complex conjugate of a complex number is the number with the same real value and imaginary magnitude, but the imaginary part is opposite in sign.

...

3.
* scale plots/images

A convolution is the result of the application of an impulse response function to an input signal.
The input signal consists of a series of samples (impulses) of various magnitudes. Each input produces a corresponding input response function

Since 'B' is composed of horizontal line samples, it appears as a horizontal broken line in the fourier transform. Conversely, the 'A' being composed of vertical lines appears as a vertical line in the fourier transform.

The diagonal crosspieces of the 'A' show up as rotated diagonals in the fourier transform, while the curving edges of the 'B' show up most as the vaguely rounded shapes at the edges of the horizontal line in the centre of the plot.


clear;
load 'p:\public\alistair\ab'
subplot(3,2,1), image(final);
colormap(gray(256));
y = fft2(final);
subplot(3,2,2), image(256*log(abs(y))/max(max(log(abs(y)))));
ydim = size(y, 1);
xdim = size(y, 2);
mask = zeros(ydim, xdim);
msize = 140;
mask(:, (xdim/2-msize/2):(xdim/2+msize/2)) = 1;

mask2 = zeros(ydim, xdim);
mask2((ydim/2-msize/2):(ydim/2+msize/2), :)=1;

y2 = mask .* y;
ya = ifft2(y2);

y3 = mask2 .* y;
yb = ifft2(y3);

subplot(3,2,3), image(abs(ya));
subplot(3,2,4), image(256*log(abs(y2))/max(max(log(abs(y2)))));

subplot(3,2,5), image(abs(yb));
subplot(3,2,6), image(256*log(abs(y3))/max(max(log(abs(y3)))));

----

lenna....

clear;
load 'p:publicalistairlenna';
subplot(2,3,1), image(xx);
colormap(gray(256));

ynew = 512;
xnew = 512;

scaled = zeros(ynew, xnew);
scaled(1:2:ynew, 1:2:xnew) = xx;

y = fft2(scaled);

subplot(2,3,2), image(256*log(abs(y))/max(max(log(abs(y)))));
subplot(2,3,3), image(ifft2(y));
mask = zeros(ynew, xnew);
msize = 128;

mask((ynew/2-msize):(ynew/2+msize),(xnew/2-msize):(xnew/2+msize)) = 1;
ya = mask .* y;

subplot(2,3,4), image(256*log(abs(ya))/max(max(log(abs(ya)))));
scaledimg = abs(ifft2(ya));
subplot(2,3,5), image(256*scaledimg/max(max(scaledimg)));