Looking carefully at those eight U.S. Army colonels from Virginia confirms that Lee’s decision was abnormal. Of those eight, seven remained loyal to their solemn oath to the U.S. Constitution. Only one colonel resigned to fight against the United States. Robert E. Lee.