The suspect attacked two Jewish passers-by before fleeing. He later wounded a border guard before he was shot dead.
Two of the Israelis were lightly wounded and the third was in a more serious condition, according to police.
Clashes broke out afterwards between stone-throwing Palestinians and police officers who used stun grenades, an AFP photographer said.
It was the second such attack in days near Damascus Gate, a main entrance to the Old City.
On Wednesday, a Palestinian woman said to be the mother of a man killed last year tried to stab Israeli police with scissors before being shot dead.
A wave of violence that broke out in October 2015 has claimed the lives of 259 Palestinians, 40 Israelis, two Americans, one Jordanian, an Eritrean and a Sudanese national, according to an AFP count.
Most of the Palestinians killed were carrying out knife, gun or car-ramming attacks, according to Israeli authorities. Others died during protests, clashes or in Israeli air raids on the Gaza Strip.
Violence has subsided in recent months, despite sporadic attacks, but the head of Israel's Shin Bet internal security service, Nadav Argaman, warned last month against complacency, saying the lull was "misleading".
A police official said the security forces would receive reinforcements in preparation for possible attacks during the week-long Passover Jewish holiday beginning April 10 when tens of thousands visit the Old City.
Eyewitness? Submit your stories now via social or:
Email: news@pulselive.co.ke
