This topic is by no means new, the idea of censuring or removing memorials to individuals deemed historically "hateful" has been in the news for months, and unlike many other media frenzies, it has not fully been supplanted by something else in the news (typically a Trump scandal these days). All over the American South, Confederate statues are coming down in the name of racial justice and harmony. But is that all there really is to this issue? Is it really just to utterly vilify historical characters simply based on one aspect of their lives? Let's consider one Confederate warrior who has come under scrutiny particularly harshly: General Robert E. Lee. Articles from new sources like the Chicago Tribune have demonized Lee as a villain who fought for slavery and that was basically the only reason he took up arms against the United States. Really? In reality, General Lee fought for the Confederacy (after refusing an offer by Abraham Lincoln to lead the Union Army) because he could not bear to go against the values of his beloved home state of Virginia. Do those values include slavery? Unfortunately, yes. Was slavery the only reason Virginia broke away from the Union? No. Other factors include economic disparities between the Northern and Southern American states as well as state sovereignty on issues like taxation. Before American Citizens call for the destruction of Lee's memorials, they should consider the fact that consider the fact that he is one of the greatest military minds America has ever produced. His tactics are still studied in military schools today and he was so good that Abraham Lincoln was willing to totally disregard Lee's slaveholding past to have him LEAD the Union, the side in the Civil War we consider to be just and good today!
The aforementioned Chicago tribune article's central thesis was that removing the statues of Confederate leaders like Lee would not lead to a slippery slope in tearing down other memorials. This is not necessarily correct at all. For example, according to an August 2017 Politifact article, activists like Al Sharpton HAVE vocalized sentiments calling for the removal of monuments to great Presidents like George Washington and Thomas Jefferson because they owned slaves. Isn't that a bit of a slippery slope? If helping found the greatest democracy of all time does not deserve memorialization, what does?
Finally, it cannot be overstated that history must be analyzed from ALL points of perspective. Nobody is perfect. Was slavery an immense sin committed by figures like Lee, Jefferson, and Washington that deserves condemnation? Of course. But did this sin define these men? Of course not. There is much more than meets the eye to complex historical analysis. Members of communities who honor these types of controversial individuals must think long and hard over whether or not they should erase the ancestors of their nation from public regard. The consequences can become ridiculous. Francis Scott Key was an infamous anti-abolitionist who used his job as a district attorney to pursue legal action against abolitionism, something he considered seditious and against American values. Should we stop singing the National Anthem? No! The contents of the National Anthem have nothing to do with Key's pro-slavery endeavors, just as Robert E. Lee's military genius had nothing to do with his practice of slavery. Think, America, think!

Source: Flickr via Creative Commons license
I can see where you are coming from - people should be defined as something more than their wrongs. I also definitely agree with your statement regarding the importance of knowing the historical context and perspectives.
With this in mind, I think it is interesting that the vast majority of Confederate statues were erected between 1890 and 1960*, which was during the Jim Crow era and Civil Rights Movement. If these statues had been erected right after the Civil War, I would believe that it was intended to celebrate these men. However, I think the time and historical context suggest that these statues may represent other values than the men themselves.
*http://www.history.com/news/how-the-u-s-got-so-many-confederate-monuments