Regression 1: More Properties

Having mentioned some of the most important properties in the previous regression post, let me discuss the properties in a more organized fashion. And, as I said before, even if you’re not using Mathematica® to do regression, you will want to find how to get many of these properties, whatever they are called by your software or if you must compute them yourself.

Let me begin by following exactly the computational sequence I laid out here as though we were trying to write code ourselves. If the following discussion seems a little too brief for you, you may want to look at that link, henceforth referred to as “the introductory post”.

I should be explicit about something: I figure that if I can’t compute something myself, then I do not understand it. Even if I can compute it, I may not understand it, but first of all, I must be able to calculate it. And that’s what I’m about to do with the regression properties: show how they are computed, and confirm that Mathematica and I get the same answers.

Happenings – 2010 Sept 25

Well, it’s that time of the week. What have I been doing and what’s happening?

I’ve continued working on sensitivity analysis in linear programming, and on internal model control. I put in a little time on group theory, and on curves and surfaces in N dimensions.

And, of course, I’ve been working on regression.

But I wouldn’t have a post ready for this Monday if I had not taken some time off from work. A friend has come in for the weekend… and while we will talk mathematics sometimes, I won’t get more than a very little mathematics done.

But this Monday’s post was brought to stage V on Wednesday. That is, all of the pictures and LaTeX have been added to the text. All I need to do is my usual polishing of the narrative, and see if there are some links I should add.

By the way, Monday’s post will be more regression properties.

Oh, readership on the blog has been picking up again. After a slow summer, I’ve been getting more than 100 hits a day for two weeks now… except for one day on which I got 97… but, then again, there was one day that got over 200. It’s nice to see.

And, the first projectiles post has gotten much more activity, so I feel more incentive to put out the second one. Unfortunately, it’s only at stage IV: the mathematics is done, but I haven’t even taken the screenshots or started creating the LaTeX.

And with that, I’m out of here for today.

Regression 1: Introduction

This is the first of several posts discussing multivariable regression (“ordinary least squares”, or OLS).

I will begin by introducing a data set which we will use for most of this set of posts… I will show you how to run a regression using Mathematica version 7 (it has changed from version 6)… I will just scratch the surface looking at the output from a regression.

Let’s get started.

setup

The following is referred to as “the Hald data”. It’s not very large — 13 observations on 5 variables.

Happenings – 2010 Sept 18

It is clear that I did not put out a post last weekend, in particular not a first regression post. Well, I tried… but it was still a fairly amorphous collection of ideas…. Today it seems clear that the first post should be about half of what I originally intended… and, furthermore, it looks like I’ll be putting out eight posts about regression.

I wish I could say that eight posts would cover the entire subject, but they will just barely get us started. Besides, while there are some simple aspects of regression that I understand, there is a lot more that I don’t understand.

Happenings – 2010 Sept 12

I’m not sure I should try writing this with a small television in my peripheral vision showing the first semi final match of this year’s US open… but I’ll try anyway. (That was yesterday.)

Let’s see. Last weekend I continued working on the next regression post. I got distracted from what I intended to write… and ended up describing the properties (computed by Mathematica®) of a regression. I had planned to introduce them piecemeal whenever I got around to discussing particular ones. Instead, I did what came naturally at the time… and begin a wholesale description.

I took it as a good omen later on in the week that someone asked on the Mathematica news group if it was a bug that “Response” held the given data y rather than the fitted values yhat. My discussion of all the properties might be useful.

Rotations: figuring out what a rotation does

I want to write a short emphatic post making one major point. I’ve made it before, but I think it needs to be hammered home.

In addition to the links in this post, you could use the Tag Cloud to find related posts.

I mentioned in this happenings post, near the end, that I had found a nice introductory book on the control of aircraft (Thomas Yechout et al., “Introduction to Aircraft Flight Mechanics…”, AIAA 2003, ISBN 1-56347-4344).

I looked at it yesterday morning. I was planning to spend the day working on the next regression post … and I did … but I figured I’d turn my kid loose first, and he wanted to play in that book.

I still think it’s a nice book… but two of their drawings for rotations are mighty confusing… at first glance, they appear to be wrong. It turns out they are right, but the display is unusual.

I’ve talked about this out before, but let me put it in a post all by itself. (This post seemed essential when I thought they had made a mistake; it is merely desirable once I saw that the reader had to take a careful look at their drawings.)

Suppose we are given the following rotation matrix…

Happenings – 2010 Sept 3

What’s been happening since last Saturday morning?

Clearly, I put out a short post about Euler angle sequences. It was quite nice that it was a short post, a couple of hours instead of half a day or an entire day. I’ll try not to worry about it the next time a post seems like it will be “too short”.

I put in a little time on linear programming… but I’m at that terrible stage where I can replicate the illustrative example, but I can’t get the very same procedure to work out on a different example. Sure, I’ll get it… but I don’t have it yet.

All it’s telling me is that I don’t actually understand the illustrative example (the story of my life).

As usual, instead of beating my head against a wall, I switched to something else. Two something elses in fact.

One, regression… I think I have decided what the next four posts will be. The issue is not the mathematics, but the organization of the material.

Two, control theory… I have moved on from “direct synthesis” to “internal model control” (IMC). I love it. It really does seem to provide a marvelous framework for my understanding. I’m looking forward to describing it.

Alas, I have not touched Boolean algebra or logic in the past week… so it may well have been three weeks now since I touched it at all.

Still, regression and control theory have come along nicely. Last weekend was very good.

I still plan to hit the Highland Games later this morning, so I may not get much math done today. But there are two more IMC examples to work out in one book… and several more in another…. Let me at them!

At times like this, my kid and my grownup agree on what we’re going to do.