Rice forced a shot in at the near post in the sixth minute of added time and Jesus then scored a sublime goal on the break to seal the three points.
United were left crestfallen especially as they had what they thought was a late winner scored by substitute Alejandro Garnacho ruled out for offside after a VAR video check.
Marcus Rashford gave United the lead against the run of play in the 27th minute, lashing home a shot after a counter-attack, but their joy did not last long though as Martin Odegaard fired Arsenal’s equaliser less than a minute later.
Arsenal were awarded a penalty midway through the second period when Kai Havertz went down in the area but referee Anthony Taylor change his mind after checking a monitor.
Mikel Arteta’s side have 10 points from four games with United on six after already losing twice.