According to People, Jesse's first day back at work was calm, but not pleasant.
"He was very quiet," a source told the mag. "He seems very upset but also very happy to be back at work."
Jesse was also photographed still wearing his wedding ring on Saturday.
As previously reported on AccessHollywood.com, Jesse called "the vast majority" of the infidelity claims that surfaced last week "untrue and unfounded," in a statement to Access on Thursday.
On Wednesday, In Touch magazine revealed their newest cover story — titled "The Ultimate Betrayal," which centered around an interview with a woman named Michelle "Bombshell" McGee, who claimed to the mag she had an 11-month affair with Jesse, including while Sandra was shooting "The Blind Side."
"I will not dignify these private matters with any further public comment," his statement said. "There is only one person to blame for this whole situation, and that is me."
Jesse apologized to Sandra and his three children, children he was photographed taking to school on Thursday, followed by the paparazzi.
"It's because of my poor judgment that I deserve everything bad that is coming my way," he continued in the statement. "This has caused my wife and kids pain and embarrassment beyond comprehension and I am extremely saddened to have brought this on them. I am truly very sorry for the grief I have caused them. I hope one day they can find it in their hearts to forgive me."
And on Wednesday night, People reported that Sandra had left the home she shares with James on Monday.
On Tuesday, Sandra pulled out of the London premiere of "The Blind Side."
Related Content from AccessHollywood.com:
VIEW THE PHOTOS: Sandra Bullock: America's Leading Rom-Com Queen
VIEW THE PHOTOS: Sandra Bullock Fashion Flashback
PLAY IT NOW: Sandra's Scandal: What Should Bullock's Next Move Be?
PLAY IT NOW: Jesse James Apologizes For Rumored Cheating On Sandra Bullock