Finding relationships in data

Wednesday, Jan 20th, 2010 by Zachary Kurtz | Leave a Comment
Filed under: Markets, Mathematics


This publication has received some attention in the popular presses (see the Wired article):

In just over a day, a powerful computer program accomplished a feat that took physicists centuries to complete: extrapolating the laws of motion from a pendulum’s swings.

Developed by Cornell researchers, the program deduced the natural laws without a shred of knowledge about physics or geometry.

The research is being heralded as a potential breakthrough for science in the Petabyte Age, where computers try to find regularities in massive datasets that are too big and complex for the human mind and its standard computational tools.

The cool part is not just the physics, but the use of an evolutionary algorithm to fit deterministic equations to any data set. The point is to find relationships in real data and see if it has any predictive power.

This approach, of course, misses the stochastic/ probabilistic properties of nature but still a useful tool.

The other cool part is that the the tool is free for the public to download (windows, linux or mac).

I did some little tests to see how it works. In excel I created the simple function f(x) = x^2 / 32 + sin(x). For the x’s I took a list of automatically generated random integers from random.org and set the confidence level at .9 for all numbers.

After about 2 minutes, it came up with this:

x/32 is ~= to .03x so the program did a pretty good job, I think.

Now lets try something more interesting: inputing 2 lists of random numbers for x and y:

If the purpose of the engine is to to find relationships between raw data, it stands to reason that it could find relationships where none exist.

Lets test the meme that LA Lakers wins/loses predict stock market price changes. I used the data from Nasdaq rates (between 1987 and 2007) provided by the LA Time and also converted Lakers performance into quantitative terms. -1 represents a championship loss, +1 a championship win while 0 represents a time when the Lakers didn’t make the finals:

Lets graph the function:

According to our function, a victory is suggestive of stock market losses and a finals loss is indicated by some NASDAQ gains. Of course, the last two years of data have smashed these “predictions.”  Interestingly, our equation predicts the highest gains when the Lakers don’t make the playoffs at all. I think this is suggestive.

Santarcho-Capitalism?

Saturday, Dec 19th, 2009 by olimay | Leave a Comment
Filed under: Economics | Tags: ,

Let a Thousand Nations Bloom has an amusing holiday themed post about a certain renegade seasteader:

Flying no flag of convenience, the guy uses ice floes around the North Pole to support and cloak highly mobile capital and productive labor; with a flair for anarchy, he disregards all laws of intellectual property to create an abundance of goods that he then feels free to distribute according to a little understood moral code; his superior logistical system flagrantly disregards all national borders and crosses them with impunity…

Read the whole thing: it makes a nice point about what gifts economies of scale have given us.

Narrative Fallacy and “Why?” Questions

Monday, Nov 16th, 2009 by olimay | 2 Comments
Filed under: Bias, Rationality, Religion

Those of you who have read The Black Swan know of Taleb’s term narrative fallacy, a tendency to overweight the stories we use to summarize facts. Taleb makes the point many times that humans like to make up causal stories and treat them as true, which is one reason why people tend to ignore randomness.

To make things worse, complete stories, ones with more details tend to be more convincing. We thus not only gravitate to stories, but stories with embellishments. Are there better (and more likely to be correct) ways of satisfying our curiosity?

Andrew Gelman gives us a bit of a hint:

Many years ago, Don Rubin convinced me that it’s a lot easier to think about “effects of causes” than “causes of effects.” For example, why did my cat die? Because she ran into the street, because a car was going too fast, because the driver wasn’t paying attention, because a bird distracted the cat, because the rain stopped so the cat went outside, etc. When you look at it this way, the question of “why” is pretty meaningless.

Similarly, if you ask a question such as, What caused World War 1, the best sort of answers can take the form of potential-outcomes analyses. I don’t think it makes sense to expect any sort of true causal answer here.

Stuff White People Like on Signaling

Saturday, Nov 14th, 2009 by olimay | Leave a Comment
Filed under: Bias, Culture | Tags: ,

The humor blog Stuff White People Like is a really fun commentary on signaling, self-deception, and irrational memes. Plenty offensive, but I see it as a really valuable educational tool.

Here’s a recent one I enjoyed that seems to be about Robin Hanson’s near/far biases and preference for visible signaling:

…white people in places like Los Angeles or Austin, TX will often promise to learn Spanish in hopes of being able to ask local taco stands about whether or not their carne asada is grass fed (”¿Ha leído usted Michael Pollan?”).

In order to reach this level of fluency and obnoxiousness, white people believe they must put themselves into a local immersion. This means a promise to watch only Spanish language TV, listen only to Spanish language radio, read Marquez in his native tongue, and watch foreign films with the subtitles turned off. There are some instances of white people doing this for almost a week!

When this technique is unavailable or fails, white people will immediately turn to books and computer software as a last ditch effort to make good on their promise. After about a week, most white people will give up and blame someone for their failure (”this software is terrible,” “there aren’t enough people in Portland who speak Farsi!”). But rather than discarding the books and software packaging, white people will simply put them in the most visible part of their book shelf. This allows white people to believe that they have not failed since they can resume their studies at any time until their death.

Read the whole thing!

Here’s a link to the full list.

Perception of Nature

Thursday, Nov 5th, 2009 by Zachary Kurtz | 1 Comment
Filed under: General

David_J_Balan at Less Wrong . The basic premise is:

People have always had a religious or quasi-religious reverence for nature. In modern times, some people have started to see nature more as an enemy to be conquered than as a god to be worshiped.

I would actually say the opposite is true. People have historically and especially prehistorically a fear of nature. This may look like reverence but, from what I understand, it seems likely that people deified nature not out of love and respect, but from fear.

As in – “Lightning God, I’ll sacrifice my best goat if you stop scaring me” or “Rain God, how about some rain so I don’t starve.”

People today have never had a greater understanding of the forces of nature and been as immune to weather effects, in all but extreme cases. Since the hippie movements, and continuing today with the green movements, I don’t think you’ve ever seen a more profound respect for nature from the general public, and have responded by demanding our institutions take measures to protect nature.

We understand that economic growth relies on using nature’s resources, but have people ever before have concluded on such a grand scale that nature needs to be protected? Property rights have been used for this purpose for centuries now.

You can certainly debate the efficacy and the intentions of the political class in regards to environmental legislation, but the motives of the general public seem relatively clear.

I don’t think the reasons are altruistic, but maybe environmentalism is the newest form of signaling status… and it’s cheap too. Becoming an environmentalist only requires the purchase of a few buttons and demanding that other people do the changing.

Emergency!

Thursday, Oct 29th, 2009 by Zachary Kurtz | 1 Comment
Filed under: Government

Gov. Paterson has declared NY to be in a state of Emergency!

The reason?

A total of 75 people have died from swine flu (in a state of 18 million).

In contrast, 1,429 people died on NY roads in 2005.

What is Paterson’s purpose other than freak people out?

Predatory Pricing

Thursday, Oct 29th, 2009 by Zachary Kurtz | Leave a Comment
Filed under: Economics

Greg Mankiw speculates about predatory pricing and antitrust policy.

Has there ever been an actual example where predatory pricing has been used to clear out the competition and then take over the market? It seems to me that there is too much information asymmetry to actually pull this off. In addition, monopolies are either efficient and price low (so who cares) or other players come into markets. So its never actually a problem, just a theoretical hyperbole that economists like to muse about and politicians like to drum up scare about to increase their own influences.

The so-called predatory price cuts have not spilled over to the sale of textbooks. Is that good news for students (as the Booksellers’ argument suggests) or bad news?

Of course its bad news to the corporations.. So-called “predatory pricing” keeps prices low. So price wars only hurt the profits of the big publishers. Everyone else wins. Any bets on when they’ll start calling for price controls? you know… to “correct” for the market?

Narrative fallacies abound

Thursday, Oct 29th, 2009 by Zachary Kurtz | Leave a Comment
Filed under: Economics | Tags:

It should be obvious what’s wrong with this quote from the BBC:

The US economy grew at an annual pace of 3.5% between July and September, its first expansion in more than a year.

The growth was helped by a substantial government spending plan, including a scrappage scheme to boost car sales.


Post hoc ergo propter hoc anyone? That’s some pretty shoddy journalism. Of course everyone, especially the White House, is going to buy the story. But, of course, if you didn’t see the recovery it wouldn’t mean that the stimulus didn’t help.

Watering the Germans

Friday, Oct 23rd, 2009 by Zachary Kurtz | 1 Comment
Filed under: Economics

From the BBC

A group of rich Germans has launched a petition calling for the government to make wealthy people pay higher taxes.

The group say they have more money than they need, and the extra revenue could fund economic and social programmes to aid Germany’s economic recovery.

This is just insane. Why don’t they invest or donate their money if they really want to help recovery?

Seems like they’re overestimating the “G” aspect in the Keynesian C + I + G + X − M = Y(GDP) formula.

What’s in a name

Friday, Oct 23rd, 2009 by Zachary Kurtz | Leave a Comment
Filed under: Economics | Tags:

Bob Murphy might be being somewhat tongue and cheek here, but he has a good point. The very title of Austrian economics makes little sense. Very few Austrian-nationals are still studying it (its based in the US now) and many of the actual original Austrians (from the Vienna school) did some of their most important work in England or America.

This amusing story demonstrates the point:

And then Stephan Kinsella relays this anecdote about Tom Woods’ debate with ING Chairman Tom McInerney (HT2 Bob Roddis):

McInerney had mentioned that Bernanke was a diligent and knowledgeable student of the Great Depression. So, when it came time for the Q&A, one audience member asked Woods to briefly explain the Austrian view of Great Depression and how it might differ from Bernanke’s view. After Woods did this, McIerney took the stage, and as if he were about to unload a devastating blow against Woods, said to him, “this might seem like a bit of an attack. Don’t take it too personally.” And then…. he began to rant about … the relatively small size of the country of Austria. I kid you not.

Some audience members began to laugh; others cringed, as McInerney dug his hole deeper while under the illusion that he was unleashing a deadly zinger. Woods kept trying to stage whisper that Austria had nothing to do with the school of Austrian economics, but McInerney, undeterred, plowed on. Thus, when Woods took the stage he said, “this might seem like an attack, but don’t take it too personally…” And then Woods commented that we may as well say we shouldn’t listen to Milton Friedman, since the GDP of Chicago is pretty low.

On the other hand, maybe if we can make Austrian economics obscure enough, we can make all opponents look like fools during debates!

Actually, the more important thing is to become mainstream enough that the average economics-interested person knows at least something of your ideas before-hand.

That is, if you care about Austrian economics at all.

I wonder if the post-Keynesians have this problem at all.

« Older Entries