May 05, 2014 12:10 PM EDT By Russell Westerholm, UniversityHerald Reporter

Condoleezza Rice's appointment as Rutgers' commencement speaker angered a group of students and faculty, but her withdrawal has upset a whole other one.

According to the New Jersey Star-Ledger, the Rutgers College Republicans led a group of student organizations in writing a letter to Robert Barchi, the university president. The letter expressed concern that a "hostile campus environment" forced Rice to rescind the invitation to speak at commencement.

Last week, the Ledger reported, students protested Rice's appointment by sitting-in at the school's administrative building near the staircase leading to Barchi's office. Along with some faculty members, the protestors disagreed with Rice's involvement in the George W. Bush Administration's invasion of Iraq.

Shortly after the demonstration, Rice declined the invite to speak at commencement, an engagement that would have paid her $35,000. Now a new set of students are upset.

"An overwhelmingly number of the students were disappointed in Condoleezza Rice no longer being the commencement speaker after a small minority of the student body and intolerant faculty members at Rutgers University protested loudly over the past month," Donald Coughlan, a Rutgers student and chairman of the New Jersey College Republicans, wrote in the letter.

Rice said in a statement that commencement is a celebration and she did now want to become a "distraction for the university community at this very special time."

"I am honored to have served my country. I have defended America's belief in free speech and the exchange of ideas. These values are essential to the health of our democracy. But that is not what is at issue here," she said. "As a Professor for thirty years at Stanford University and as (its) former Provost and Chief academic officer, I understand and embrace the purpose of the commencement ceremony and I am simply unwilling to detract from it in any way."

Barchi and the Rutgers Board of Governors at no point seemed willing to disinvite Rice from speaking at the ceremony. However, the university president had no choice but to accept her withdrawal.

Rutgers has not commented on the letter led by the College Republicans nor has it announced a new commencement speaker.

See Now: Covert Team Inside Newsweek Revealed as Key Players in False Human Trafficking Lawsuit