Skip to content

Assessing research quality

Academic research is difficult to evaluate. In order to know the significance of an article, a result or an experiment, one must know a lot about the relevant field. It is probably fair to say that few people read research articles in great depth unless they work in exactly the area the article is in. PhD theses might cite hundreds of articles, but it seems natural that not all of these articles will be read with the same degree of scrutiny by the author of the thesis.

Hence the trouble with obtaining funding for research. In order to obtain funding, you have to communicate something that seems incommunicable without the full commitment of the reader. Grant dispensers want to know a number on a scale: “what’s the quality of this paper between 0 and 1?”, but this quality number cannot be communicated separately from the full substance of the paper and its environs. And thus we end up with keywords, catchphrases that become associated with quality for short periods of time, as a way of bypassing this complexity, an approximate way of indicating that you are doing research on something worthwhile.

This reflects a broader problem in society of evaluating authorities. I cannot evaluate my doctor’s, or my dentist’s, or my lawyer’s work, since I don’t have the necessary competence. Accordingly, I base my trust on the person and some of their superficial attributes, instead of judging the work by itself. It seems that the same kind of thing becomes necessary sometimes in choosing what researchers to fund.

It also points to a faculty that must have evolved in human being since millennia: the capacity for evaluating important properties of things we do not understand well very quickly, for danger, nutrition, etc. Only that this faculty does not translate well to research…

Posted 28 April 2011 † johan § Computer science ‡ Comments (0) ° Tagged: , , , ,

Objective and subjective reality; perspectivism

Nietzsche rejects the idea of an objective reality. He appears to give a generative status to the faculty of interpretation, in effect saying that the subject creates the world through her interpretations. Simultaneously, he champions the “intellectual conscience” and the value of scientific method and inquiry. How to make sense of this apparent contradiction?

It might be thought at first that the assertion that all judgments are subjective has some exceptions. After all, maybe we all agree that matters of taste and style are inherently more subjective than measurements of the length of a pencil or the weight of a stone. Maybe we would be tempted to posit a hierarchy of degrees of subjectivity. But Nietzsche rejects this too, emphatically expressing that there is no objective basis to which observations can be reduced, no judgment that is absolutely and irreducibly validated. For Nietzsche, the world seems to consist of multiple interlocking interpretations that support each other, a bit like an M. C. Escher drawing.

Elsewhere, Nietzsche invokes the death of God. Christoph Cox, in his “Nietzsche: Naturalism and Interpretation”, points out that the death of God as an idea has only been understood in its most shallow form if it is seen as a mere rejection of Christianity. For Nietzsche, Platonism, “the thing in itself”, the “forms”, “truth”, “paradise” and “objective facts” are all — maybe paradoxically — ways of rejecting reality, rejecting the world. They are dogma. The death of God, Cox asserts, is the end of all these various forms of dogma, not just of Christianity.

As for the intellectual conscience, Cox asserts that by this, Nietzsche simply means that one must question and attack one’s perspectives and interpretations as much as possible, and that a refusal to do this — an acceptance of dogmatic thought — would be a betrayal of the intellectual conscience. In this view, Nietzsche seems to state that in order to best know the world, we must entertain multiple parallel perspectives and harden each one as much as possible through questioning.

A naive questioning of objective truth can lead to a naive relativism, in which every assertion appears to be equally valuable or equally true. It is often on this account that social philosophers and thinkers of today are criticised, as champions of a destruction and levelling of all valuation, a mindless relativism. However, the idea of the intellectual conscience does seem as if it can point the way to new and quite concrete valuation. Nietzsche’s project is ultimately a constructive one which seeks to show a way forward.

What of science then, and its claims to empirical, objective truth, found through experiment and measurement? It seems that scientific thought and scientific findings are in no way invalidated through a Nietzschean epistemology. Science would merely have to forge relationships with other perspectives and find useful ways of relating to them, instead of claiming to be the sole valid way of viewing the world.

After all, what evidence is there that the world exists objectively and independent of the mind? And if there is no evidence either way, let us use Occam’s razor. Which alternative is the simplest explanation?

Posted 31 March 2011 † johan § Philosophy ‡ Comments (3) ° Tagged: ,

Japan earthquake: 17 March

Six days have passed since the fateful earthquake and tsunami of 11 March. I’m still staying in Tokyo, and unlike a lot of the foreigners here, I don’t feel that there is any immediate danger to my person.

Life goes on: Yesterday I had a few drinks with my friend. A lot of places were closed, but some bars are still open. There’s both fewer customers and less electricity to go around at the moment. There was an air of calm bravery and defiance among the customers at the bar.

Some books I had ordered from Amazon arrived this morning. I used the trains as usual to go to the lab, where I worked on my research. This evening, before I left the lab, power saving efforts were intensified, since it’s been getting very cold the past few days, and more electricity is needed for simple heating. Before I left the lab I looked out the lab window. I had never before seen the skyscrapers in Marunouchi, around the Imperial Palace, so dark at night.

The situation is not completely under control yet. The big story today has been about the attempts to cool the reactors with water. Because of intense radiation immediately around the plants, it’s necessary to approach the plants by helicopter and dump water from above. Lately, they have also tried shooting water at the plants from trucks at a distance.

Even though the general concern is rising even among Japanese people, the  “foreign consensus” and the “Japanese consensus” are still strikingly different. Many countries are arranging tickets for people to go out of Japan, and even more of my friends have taken refuge in the Kansai region. This is a completely fair decision and there is no harm in taking precautions. Personally I try to keep a close watch on the radiation levels and the news. Recently many more measuring stations have been added to this geiger counter map. If the levels go up too far, I would definitely consider getting out of the Tokyo area for a while. But I consider myself attached to Tokyo, I live here, and I have projects I want to finish here. Right now, I don’t want to flee without good reason.

Since radiation is something invisible that we cannot see, smell or touch, and since nobody can go near the plants to look carefully at them by now, any understanding of the situation necessarily contains a lot of interpretation. Nobody fully knows what the current state of the plants are or what is going to happen. Clearly there are big risks. But I think it is fair to say that the stories circulated by foreign media do not reflect the situation in most of Japan. Those near Sendai or near the power plants have suffered terribly and are facing great risks. But the vast majority of Japan is experiencing little outside of power shortages. Those who have relatives in Japan should take this into account when they read the news.

My friend Jacob Ehnmark, who used to live in Sendai, is blogging about his escape from Japan and his impressions here.

Posted 17 March 2011 † johan § Life ‡ Comments (1) ° Tagged: ,

Japan earthquake: situation as of today

I thought I would write a quick summary of the situation following the huge earthquake in Touhoku, as I understand it, possibly with more updates to follow.

1. I am fine, and there is no immediate danger to me personally. When the quake occurred, I was in Sapporo, and I came back to Tokyo on Saturday night.

2. There was huge devastation in the affected area in Iwate-ken, Miyagi-ken, Sendai and so on. Many people are still missing in those areas.

3. Ever since the tsunami, Tokyo Electric Power (TEPCO) has struggled hard to  cool down its nuclear power plants in Fukushima. Here is one of many articles that summarise the situation. Useful graphics from Washington Post. Various incidents have happened, such as explosions (apparently not in the reactor core, but as a result of hydrogen and oxygen being released) and a fire in one reactor, which has now been extinguished. Elevated radiation levels have been measured near the reactors and people in a 30 km radius are asked to stay indoors or evacuate.

3.5. The water level inside some of the reactors’ so-called second containments was too low at certain points, exposing the fuel rods, and it is likely that fuel inside was melting. Supposedly, even if it melts, it is meant to be collected in a container underneath the core.

4. Tokyo is 250 km from Fukushima, and the radiation levels are far lower here. Some Geiger counters (in Tokyo) are available online (also on map). While the radiation levels near the reactor are now at dangerous levels, in Tokyo, so far they have not gone above harmless levels.

5. Accounts of the disaster vary widely, from careful, measured fact reporting to doomsday predictions. Foreign media are generally playing up the disaster angle quite a lot. For example, this article from Der Spiegel. What is reported in Japanese media still sticks mainly to observed facts and public advisories, although there is a sense of increasing exasperation with the lack of information from TEPCO and the government.

6. If the radiation level keeps going down, the biggest risks to us living in Tokyo should be the risk of a strong aftershock following the initial quake, especially if there is an associated tsunami. However, as time passes, this risk is written down.

7. Across Japan, people are trying hard to conserve electricity because so many power plants have been shut down. Parts of east Japan are having a controlled power outage on a rotational basis. Trains run less frequently than usual, but as of now, in central Tokyo, daily life is basically still normal.

Posted 15 March 2011 † johan § Life ‡ Comments (0) ° Tagged: ,

JTronic, a programming game

Now for something quite different from the philosophical entries I’ve been writing recently. We finally got the time to release the game “JTron” on Sourceforge. It is there under its new name Jtronic.

This game is designed for programming competitions; participants program an agent that plays the game by obtaining information about its environment and making decisions. The programming language is Java. The game itself combines features of Pac-man and Tron in what we think is an interesting synthesis.

This game was used for the ACM-ICPC regional contest in Tokyo 2010, for the “Java Challenge” part. I’ve had the honour to work with some great people in the Honiden lab in developing this game, and we spent time on it on and off for almost a year. A very educational process, and the final result was not bad at all.

If you can program in Java, you can have fun with this framework by programming your own agent and competing against others (including some pre-made ones that are included in the release), or you can contribute to the game framework itself, should you feel so inclined. It’s GPL licensed. The documentation is still slightly sparse, but we will release more “shortly”.

A similar framework is the “Icy Challenge” that was used in the world final ICPC Java Challenge. It has much nicer graphics than our game. Maybe someone would take the time to make 3D graphics for JTronic?

Posted 03 March 2011 † johan § Computer science § Software development ‡ Comments (0) ° Tagged: , ,