The Confederate Flag is much like the Swastika, they both had different meanings in the beginning. The Swastika actually meant "good" and "peace", however the Nazis and the Holocaust changed that.
BTW you do know that the Confederate Flag was only a battle flag right? It was never a national flag, the Stars and Bars was. However the battle flag was used as the union (top left part of the flag) in later versions of the national flag.
I'd say that we could partially blame President Lincoln for the racism that's now associated with the Battle Flag and the Civil War in general. IMO he used the "free the slaves" for political reasons. The other thing to blame is our school system.