What is the difference between single-quoted and double-quoted strings in PHP?

PHP strings can be specified not just in two ways, but in four ways.

1. Single quoted strings will display things almost completely "as is." Variables and most escape sequences will not be interpreted. The exception is that to display a literal single quote, you can escape it with a back slash \', and to display a back slash, you can escape it with another backslash \\ (So yes, even single quoted strings are parsed).
2. Double quote strings will display a host of escape sequences (including some regexes), and variables in the strings will be evaluated. An important point here is that you can use curly braces to isolate the name of the variable you want evaluated. For example let's say you have the variable $type and you want to echo "The $types are". That will look for the variable $types. To get around this use echo "The {$type}s are". Take a look at string parsing to see how to use array variables and such.
3. Heredoc string syntax works like double quoted strings. It starts with <<<. After this operator, an identifier is provided, then a newline. The string itself follows, and then the same identifier again to close the quotation. You don't need to escape quotes in this syntax.
4. Nowdoc (since PHP 5.3.0) string syntax works essentially like single quoted strings. The difference is that not even single quotes or backslashes have to be escaped. A nowdoc is identified with the same <<< sequence used for heredocs, but the identifier which follows is enclosed in single quotes, e.g. <<<'EOT'. No parsing is done in nowdoc.

Notes: Single quotes inside of single quotes and double quotes inside of double quotes must be escaped:

$string = 'He said "What\'s up?"';
$string = "He said \"What's up?\"";

Speed:
There is no difference.
Please read a credible article on the matter from one of PHP core developers. Speaking of tests, one should never take them for granted. It must be understood that writing a credible test and, especially, interpreting its results requires a lot of knowledge and experience. Which means that most tests out there are just bogus. For example, in a code like this

for($i=0;$i<100000;$i++) {
    'string';
}

The quoted string gets parsed only once, along with entire script, and then gets translated into opcode. Which is then gets executed a million times. So it measures anything but parsing. And that's only a tip of the iceberg. For a nanobenchmark like this, it's practically impossible to create a credible test that wouldn't be spoiled by some interfering side effect.