Chapter 17 · Mastery

Cheat Sheet & Practice

The whole course on a few screens, a decision procedure for the questions you will be asked, and a mixed problem set to prove to yourself that you have reached a solid level.

⏱ 2–3 h for all problemsEverything

The models at a glance

ModelDefinitionMemoryEquivalent to
DFA$(Q,\Sigma,\delta,q_0,F)$, $\delta: Q\times\Sigma \to Q$finiteNFA, ε-NFA, regex, right-linear grammar
NFA$\delta: Q\times\Sigma_\varepsilon \to \mathcal{P}(Q)$finiteDFA (subset construction, up to $2^n$ states)
Regex$\emptyset, \varepsilon, a, \cup, \cdot, {}^*$—NFA (Thompson), from NFA by state elimination
CFG$(V,\Sigma,R,S)$, rules $A\to w$—PDA; normal forms CNF, GNF
PDA$\delta: Q\times\Sigma_\varepsilon\times\Gamma_\varepsilon \to \mathcal{P}(Q\times\Gamma^*)$one stackCFG (DPDA strictly weaker)
LBATM restricted to input cellslinear tapecontext-sensitive grammars
TM$\delta: Q\times\Gamma \to Q\times\Gamma\times\{L,R\}$unbounded tapemultitape, NTM, 2-stack PDA, RAM, unrestricted grammars, every programming language

Constructions you must know cold

ConstructionInput → OutputKey ideaSize
Product2 DFAs → DFA for ∩ / ∪run in parallel, states are pairs$|Q_1|\cdot|Q_2|$
ComplementDFA → DFAswap $F$ and $Q\setminus F$ (complete DFA!)same
Subset constructionNFA → DFAstates = ε-closed sets of NFA states$\le 2^{|Q|}$
Thompsonregex → ε-NFAstructural induction, glue with ε$O(|R|)$
State eliminationNFA → regex$R_{pq}R_{qq}^*R_{qr}\cup R_{pr}$can be exponential
Table fillingDFA → minimal DFAmark distinguishable pairs backwards$O(n^2|\Sigma|)$ (Hopcroft: $n\log n$)
CNF conversionCFG → CNFSTART, DEL, UNIT, TERM, BINpolynomial
CYKCNF + $w$ → yes/noDP over substrings$O(n^3|G|)$
CFG → PDAgrammar → PDAstack holds unmatched sentential form3 states
Universal TM$\langle M,w\rangle$ → simulaterulebook on a tape—

Is $L$ regular? A decision procedure

  1. Is it finite? Then yes.
  2. Does recognizing it need only bounded memory? (parities, remainders, last $k$ symbols, “seen X yet”) → build a DFA/NFA/regex. Done.
  3. Does it need to compare two unbounded quantities (counts, a string with its copy/reverse)? Suspect non-regular. Then:
    • Try closure: intersect with a regular language / apply a homomorphism to reach $0^n1^n$.
    • Or Myhill–Nerode: find infinitely many pairwise distinguishable prefixes (e.g. $0^i$ distinguished by $1^i$).
    • Or pumping: pick $s$ in terms of $p$, with the “interesting” part within the first $p$ symbols; handle all splits; choose $i$ (remember $i = 0$).

Is $L$ context-free?

  1. One matching / nesting (balanced, $a^nb^n$, palindromes, $a^ib^jc^k$ with $i=j$ or $j=k$)? → write a grammar or a PDA.
  2. Two independent matchings, or crossing dependencies ($a^nb^nc^n$, $ww$, $a^ib^jc^id^j$)? Suspect not. Then:
    • Closure: intersect with a regular language to reach $a^nb^nc^n$.
    • CFL pumping: $s = uvxyz$, $|vxy| \le p$ — the window can’t touch three blocks of length $p$.
    • Unary alphabet? Then context-free ⇔ regular.

Is a problem decidable?

  1. Is it about a finite object or a bounded run? (a DFA, a CFG’s membership/emptiness, “within $k$ steps”) → usually decidable: simulate or search a finite space.
  2. Is it about the language / behaviour of an arbitrary TM or program? → Rice: nontrivial ⇒ undecidable.
  3. Otherwise reduce from $A_\text{TM}$, $\textit{HALT}_\text{TM}$, $E_\text{TM}$ or PCP: assume a decider for your problem, build a decider for the known one. Construct (don’t run) a machine whose language encodes the answer.
  4. Recognizable? If “yes” instances have a finite certificate you can search for, yes. If both the problem and its complement are recognizable, it is decidable. Unrecognizable: mapping-reduce from $\overline{A_\text{TM}}$.

Is a problem NP-complete?

  1. Show it is in NP: name the certificate and give a polynomial verifier.
  2. Pick a known NP-complete problem $C$ that “looks similar” (3SAT for logic, CLIQUE / VERTEX-COVER / INDEPENDENT-SET for graphs, SUBSET-SUM for numbers, HAMPATH for orderings, 3-COLOR for partitions).
  3. Give a polynomial $f$ with $x \in C \iff f(x) \in$ your problem. Prove both directions of the iff.

Closure & decidability summary

∪∩complement·*membershipemptinessequivalence
Regular✓✓✓✓✓✓✓✓
Context-free✓✗✗✓✓✓✓✗
Decidable✓✓✓✓✓✓✗✗
Recognizable✓✓✗✓✓✗✗✗

The ten ideas to remember

  1. A problem is a language; solving it is deciding membership.
  2. A state is a summary of the past that is sufficient for the future.
  3. Nondeterminism = guessing = tracking all possibilities at once.
  4. Long inputs + finite memory ⇒ repetition ⇒ pumping (pigeonhole).
  5. Minimal DFA states = classes of indistinguishable prefixes (Myhill–Nerode).
  6. Recursion/nesting needs a stack; one stack matches one thing.
  7. Two stacks, or a rewritable tape, give universal computation (Church–Turing).
  8. Self-reference + diagonalization ⇒ the Halting Problem is undecidable.
  9. Reductions transfer hardness from left to right: $A \le B$, A hard ⇒ B hard.
  10. Easy to check ≠ easy to find? That is P vs NP.

Mixed practice problems

Try each one for at least 10 minutes before opening the solution. Classify, design, or prove as appropriate.

Design a DFA over $\{a,b\}$ accepting strings with an even number of $a$’s and ending in $b$. How many states? easy

Product of “parity of $a$’s” (2 states) and “last symbol is $b$” (2 states): 4 states $(E/O, b?/\neg b)$. Start $(E, \neg b)$; accept $(E, b)$. All four states are reachable and distinguishable, so 4 is minimal.

Is $\{a^m b^n : m + n \text{ is even}\}$ regular? easy

Yes: $(aa)^*(bb)^* \cup a(aa)^*b(bb)^*$. Only the parity of the total length matters, plus the order constraint $a^*b^*$.

Is $\{a^m b^n : m \ne n\}$ regular? Context-free? medium

Not regular: if it were, its complement intersected with $a^*b^*$, namely $\{a^nb^n\}$, would be regular. Context-free: $S \to aSb \mid A \mid B$, $A \to aA \mid a$, $B \to bB \mid b$ (generate a balanced core, then extra $a$’s or extra $b$’s).

Convert the NFA with states $\{1,2\}$, start 1, accept $\{2\}$, $\delta(1,a)=\{1,2\}$, $\delta(2,b)=\{1\}$ to a DFA. easy

$\{1\} \xrightarrow{a} \{1,2\}$, $\{1\} \xrightarrow{b} \emptyset$; $\{1,2\} \xrightarrow{a} \{1,2\}$, $\{1,2\} \xrightarrow{b} \{1\}$; $\emptyset$ loops. Accept $\{1,2\}$. The language is $a(a \cup ba)^*$: start with $a$, every $b$ is followed by an $a$, end with $a$.

Prove $\{a^ib^jc^k : i \lt j \lt k\}$ is not context-free. medium

Take $s = a^pb^{p+1}c^{p+2}$. Since $|vxy| \le p$, the window never contains both $a$’s and $c$’s. If $v$ or $y$ contains two kinds of symbols, pumping up puts symbols out of order. Otherwise let $\alpha, \beta, \gamma$ be the numbers of $a$’s, $b$’s, $c$’s in $vy$:

  • $\alpha \ge 1$ (so $\gamma = 0$): pump up ($i = 2$). If $\beta \ge 1$ the $b$-count becomes $\ge p+2 = k$; if $\beta = 0$ the $a$-count becomes $\ge p+1 = j$. Either way $i \lt j \lt k$ breaks.
  • $\alpha = 0$, $\beta \ge 1$: pump down ($i = 0$); the $b$-count drops to $\le p$, the $a$-count.
  • Only $c$’s: pump down; the $c$-count drops to $\le p+1$, the $b$-count.
Give a CFG for $\{a^ib^jc^k : i + k = j\}$. medium

$j = i + k$ means $b^j = b^ib^k$: $S \to XY$, $X \to aXb \mid \varepsilon$, $Y \to bYc \mid \varepsilon$.

Minimize: states $A..D$, start $A$, accept $\{C, D\}$; $A\xrightarrow{0}B, A\xrightarrow{1}C, B\xrightarrow{0}A, B\xrightarrow{1}D, C\xrightarrow{0,1}D, D\xrightarrow{0,1}C$. medium

Pass 0 marks $\{A,C\},\{A,D\},\{B,C\},\{B,D\}$. $\{A,B\}$: on 0 → $\{B,A\}$, on 1 → $\{C,D\}$. $\{C,D\}$: on 0,1 → $\{D,C\}$. Neither ever gets marked: $A \equiv B$ and $C \equiv D$. Minimal DFA: 2 states — the language is “contains at least one 1”.

Is $\{\langle M\rangle : M \text{ accepts some palindrome}\}$ decidable? Recognizable? medium

Undecidable by Rice (“$L(M)$ contains a palindrome” is a nontrivial language property). Recognizable: dovetail $M$ over all palindromes (run $i$ steps on each of the first $i$ palindromes, for $i = 1, 2, \dots$); accept when one is accepted.

Is $\{\langle G\rangle : G \text{ is a CFG and } L(G) \text{ contains some palindrome}\}$ decidable? hard

Undecidable, by reduction from PCP. Given dominoes $[t_i / b_i]$, build $G$ with rules $S \to t_i\,S\,b_i^R \mid t_i\,\#\,b_i^R$ for each $i$ ($\#$ a fresh symbol). $G$ generates exactly the strings $t_{i_1}\cdots t_{i_n}\,\#\,(b_{i_1}\cdots b_{i_n})^R$. Such a string is a palindrome iff the part before $\#$ is the reverse of the part after it, i.e. iff $t_{i_1}\cdots t_{i_n} = b_{i_1}\cdots b_{i_n}$ — a match. So a decider for this problem would decide PCP.

Is $\{\langle M\rangle : M \text{, started on a blank tape, eventually writes the symbol } 1\}$ decidable? Does Rice’s theorem apply? hard

Rice does not apply — this is a property of the machine’s behaviour, not of $L(M)$. It is still undecidable, by reduction from $A_\text{TM}$: given $\langle M, w\rangle$, build $M'$ that writes $w$ on its tape using a renamed copy of $M$’s alphabet containing no $1$, simulates $M$ on $w$ (never writing $1$), and writes a $1$ only if $M$ accepts. Then $M'$ writes a $1$ iff $M$ accepts $w$.

Show DOUBLE-SAT (formula has at least two satisfying assignments) is NP-complete. medium

In NP: certificate = two distinct satisfying assignments. Reduction from SAT: given $\varphi(x_1..x_n)$, output $\varphi' = \varphi \land (y \lor \bar y)$ with a fresh variable $y$. Each satisfying assignment of $\varphi$ gives two of $\varphi'$ (any $y$), and $\varphi'$ is satisfiable only if $\varphi$ is.

True or false: “If $A \le_p B$ and $B \in \text{NP}$, then $A \in \text{NP}$.” And: “If $A \le_p B$ and $A$ is NP-complete, then $B$ is NP-complete.” medium

First: true (compute $f(w)$, then verify for $B$). Second: false — $B$ is NP-hard, but it might not be in NP (e.g. $B = \textit{HALT}_\text{TM}$, to which SAT reduces).

A language $L$ and its complement are both context-free. Must $L$ be regular? Must it be deterministic context-free? hard

Neither. Not regular: $L = \{a^nb^n\}$ — its complement (strings not of the form $a^*b^*$, plus $a^mb^n$ with $m \ne n$) is context-free, but $L$ is not regular. Not deterministic: $L = \{ww^R : w \in \{a,b\}^*\}$ — its complement (odd-length strings, plus even-length non-palindromes: $S \to aSa \mid bSb \mid aXb \mid bXa$, with $X$ generating every even-length string) is context-free, yet $L$ is not a DCFL.

Where to go next

ResourceWhy
Michael Sipser, Introduction to the Theory of ComputationThe clearest textbook; this site follows its notation closely.
Hopcroft, Motwani & Ullman, Introduction to Automata Theory, Languages, and ComputationMore algorithms and detail on automata and grammars.
Dexter Kozen, Automata and ComputabilityShort, elegant lectures; great second pass.
Arora & Barak, Computational Complexity: A Modern ApproachThe next step after Chapter 15.
MIT 18.404J (Sipser’s lectures, MIT OpenCourseWare)Free video lectures of the full course.
Aho, Lam, Sethi & Ullman, Compilers (the Dragon Book)The theory applied: lexers, parsers, analysis.
You made it

If you can do most of the problems above, you understand the theory of computation at the level of a strong undergraduate course: you can design machines, convert between models, prove both possibility and impossibility, and recognize these ideas in real systems. Go mark the chapters you have mastered — and come back to the Intuition boxes whenever you need a refresher.