Caraway is mayor pro tem, an honorary post granted by fellow council members that puts him first in line to become interim mayor until the new mayor elected in May takes office.
Political opponents are lobbying council members to stop Caraway from becoming mayor.
Caraway has been the subject of many controversies during four years on the City Council.
Most recently, he drew fire for giving Philadelphia Eagles quarterback Michael Vick a key to the city of Dallas, despite Vick's conviction on dog fighting and animal cruelty charges.
"The judgment of Mayor Pro Tem Caraway is called into question by his decision to celebrate Michael Vick," resident Valerie Kessner told the City Council Wednesday.
Mayor Tom Leppert said the key Caraway awarded was not an official Dallas key, which only the mayor can grant. But Leppert said he supports new rules on how keys are awarded to avoid such controversies in the future.
"Clearly, some of those actions I find frustrating, as I think an awful lot of people do," he said.
But Leppert sidestepped the issue of whether Caraway should remain mayor pro tem and denied that the timing of his possible announcement to step down is a factor in whether Caraway becomes mayor after he is gone.
"The council makes that decision," he said. "If the council wants to make a change, they're free to do that, but it's up to the council. I can't do anything about that."
Councilman Ron Natinsky, the only member to announce he is running for mayor in May, said he also supports new rules about granting keys.
"I think it was an embarrassing thing for the city to be put in the limelight on that," he said. "It's very unfortunate."
But Natinsky stopped short of calling for Caraway's ouster as mayor pro tem.
"At this point, there doesn't seem to be a volition on the part of the council to get the necessary signatures to move it on to a vote," he said.
Caraway has said he will not run for mayor in May but will run for re-election to his council seat.
An effort to block him from an interim mayor position that would only last several months could cause hard feelings.
DallasSouthNews.org editor Shawn Williams said supporters of such an effort would feel repercussions from Caraway supporters.
"There are a lot of people in the African-American community who support giving ex-offenders a chance, who support where Mayor Pro Tem Caraway's heart was," he said.
Williams said Leppert should make an announcement about his future soon to help his campaign but also for the city's benefit.
"The sooner he does it, the better for the city of Dallas, because the city can begin to move in a different direction," he said.
Caraway declined to comment Wednesday on the matter.