Models of computation

Yesterday we looked at real machines; today we'll look at theoretical machines. But they're very much related.

What sort of computation will we model?

We'll worry about recognizing a "language"; in this context, that means a set of "words" such as aaba drawn from some defined alphabet.

Example language: The decimal representations of multiples of three, such as 3, 927, and 1140.

We want to define a simple machine that says whether a particular word is in the language.

The finite state machine

A finite state machine is a set of states (represented by circles) and transitions (represented by arrows connecting states). One state is designated as the initial state, via an arrow pointing to a state. And states can be designated as final states, via a doubled circle.

Does this accept anteater?

   a     n     t     e     a     t     e     r
0 --> 1 --> 1 --> 1 --> 1 --> 2 --> 2 --> 2 --> 2

Does this accept aardvark?

   a     a     r     d     v     a     r     k
0 --> 1 --> 2 --> 2 --> 2 --> 2 --> 3 --> 3 --> 3

What types of strings does this machine accept?

What types of strings does this automaton accept?

Try drawing an automaton for each of the following languages.

1. each string containing the letter sequence abb anywhere within it. Examples: abb, abaabb, bbbabbb.

2. each string with an even number of a's and an even number of b's. Examples: aaaaaa, abababab, abba.

3. each string of the form aa...abb...b, with the same number of as and bs. Examples: ab, aabb, aaabbb.

But #3 is impossible! Looks that way, but how can we be sure?

Say we have a machine M, and let n represent how many states it has. Consider the following strings.

Consider the two strings aⁱb^j and a^jb^j ( i != j ). Either both will be accepted or both will be rejected, since the machine is in the same state after handling the a part of both strings. But aⁱb^j should be rejected, and and a^jb^j should be accepted. So it must be wrong on one of these.

The Push Down Automaton

It turns out the answer is yes; what we need here is a Push Down Automaton (PDA).
A PDA is just a FSM with an (infinite) stack.
In addition to moving between states, it can also push symbols onto the stack, pop them off of the stack, and make decisions based on the symbol on the top of the stack (plus its current state). But it can never look at or change any of the symbols in the stack except the top one.

A PDA can recognize aⁿbⁿ easily, by pushing a symbol onto its stack each time it sees an a. When it sees its first b, it starts popping symbols off of the stack. If it pops off its last symbol when it sees its last b, it can succeed, because we know the number of as and bs were the same.

So the next natural question to ask is, fine, but are there languages that a PDA cannot recognize?

Well, consider aⁿbⁿcⁿ.

We can pretty quickly see that the little trick we just used won't work here. But how can we be sure that there isn't any way a PDA can do this?

This involves something called the pumping lemma, works a lot like the proof above, but gets a bit complicated. Let's just go with our intuition for now.
(We're actually ignoring lots of interesting details throughout this lecture.)

The Turing Machine

Yes. Suppose instead of an infinite stack we had an infinite tape.
In other words, we do not have to stick to the top of a stack anymore.
There is now a tape head that keeps track of the current position on the tape.
We can read and write the current position, and move left or right.

This is a Turing Machine (TM) (named for that same Turing guy again).
For any given Turing Machine, the FSM control is fixed, but you can start it up with different things on the tape.
So you design a particular TM by specifying the FSM control unit.

Turing came up with a Turing Machine FSM which could simulate any other Turing Machine, called a Universal Turing machine.

So Turing showed that a suitably programmed TM can calculate anything that can be calculated by any TM
(or, as it turns out, any of the other automata [with finite programs] that anyone has ever come up with!!)

(This sort of equivalence is proven by simulating machine 1 using machine 2, and then simulating machine 2 with the machine 1.)

This leads us to the Church-Turing Hypothesis: that a Turing Machine can compute anything that any finite mechanical method can.

Wow. And they're so deceptively simple looking.

These ideas are still in use today in real computers.

Interesting side note: real computers are finite, but we pretend they have infinite memories to keep things simple.

The Chomsky Hierarchy

Now the bad news...

Amazingly, you can prove that there are things that cannot be calculated, even by a Turing Machine!

E.g., the Halting Problem. (It turns out Turing proved this too!)

Great simple proof:

if Halts(P) then
    while (true) {}  // Hah!  This won't halt!
else
    return "Oh oh!"; // Ha ha!  This does halt!