This is a fun and interesting topic. The following may be out of left field, but i have found that when I have hired graduate students for statistical support, it did not work so well when I hired graduate students from the mathematics department. The students from the statistics or biostatistics department usually worked out quite well. I don't think the difference was that the students with a statistics background had learned the things that I needed. No, in learning new material the mathematicians did not "get it" as quickly as the statisticians did. Statistical work involves a different way of thinking than mathematical work. Neither one is better or worse than the other, they are just different. They are quite different fields because of the different kind of imagination and way of thinking involved. There is my 2.04 cents worth on the topic.
This is an interesting topic. When I introduce statistics to students for the first time, I try to use the following example to make them see the fundamental difference between math and stats:
Suppose someone asks you when should she leave for the airport if the airport is 20 miles away. What would you suggest?
The math answer is something like: if she drives at 60 miles/hour she would take 20 mins, so she can leave with 20 mins in hand.
The stats answer is something more practical: if she has 45 mins in hand we are fairly sure she would reach there in time. The "45" here is an estimate. It may be calculated from existing data (from multiple past trips to the airport – i.e. via a frequentist approach), or a mix of hunch/experience and data (i.e. via more of a Bayesian integration) using some kind of a model. The "fairly sure" part as we know can also be quantified and presented as a confidence/credible interval.
What it essentially indicates is that math is deductive logic – given certain axioms the answers just follow, whereas stats is a part of inductive logic – here the answer is derived from only a sample of data and generalized (and is inherently prone to error). Of course as others have already mentioned stats uses a heavy dose of math since there are numerous deductions that are just an integral part of it, but philosophically it is quite different from math.