Opposition candidate Hakainde Hichilema, 52, of the United Party for National Development (UPND), said some remote parts of the country had not received ballot papers halfway through the polling day.
"Why are there no ballot papers in our strongholds, someone is scheming around. It's fraud," Hakainde told reporters after casting his ballot at a school in Lusaka's affluent Kabulonga suburb.
Hichilema is tipped as the main challenger to Defence Minister Edgar Lungu, 58, who represents the ruling Patriotic Front (PF).
At stake are the remaining year and a half of Sata's five-year term in the copper-rich southern African nation.
Election-weary Zambians, who voted in scheduled elections that brought Sata to power three years ago and are also due to cast ballots next year, formed long queues despite early morning cold weather.
The rivals -- Lungu the lawyer and Hichilema the businessman, known as HH -- both drew huge crowds at last-minute rallies, but in the absence of opinion polls analysts hedged their bets.
"It's a two-horse race," said Oliver Saasa, CEO of Premier Consult, a business and economic consultancy firm. "It's quite clear this is a very closely run race."
In Lusaka's Kanyama working class suburb, excited voters applauded and ululated when a presiding officer declared the crowded polling station open.
"My vote is going to make a difference, we are going to remove this ...(PF) family," said 55-year old vegetable vendor Matron Siyasiya. "They can claim all the good work, but God's favour is on my candidate, and that is HH."
But Grace Nyirongo, who runs a food take-away business said she was satisfied with the government and echoed the ruling PF's campaign slogan of continuity.
"We want the government to continue with the projects started by Sata. Frankly there's no need to start afresh," said Nyirongo.
Shortly after the polls opened it began raining heavily in Lusaka, but that did not deter the voters.
Standing in rain-drenched clothes on muddy ground, with no umbrella or raincoat, PF supporter Allan Kabwe's spirits could not be dampened.
"I know many people will be discouraged, but after I finish voting, I am going door to door to encourage people to come and vote. We have to put Edgar into state house," said the 24-year-old street vendor.
"I hope UPND supporters fail to come."
Hichilema's camp is seen to have received a boost from the infighting within another major opposition party, the Movement for Multiparty Democracy (MMD), whose candidate Nevers Mumba is given little chance.
Lungu's Patriotic Front went into the vote badly fractured by a bitter power struggle after Sata's death in October, just three years into his five-year term.
Two opposing camps -- one led by Lungu and another by interim president Guy Scott -- nominated rival presidential candidates.
After many weeks of mud-slinging, Lungu emerged as the sole candidate -- but of a weakened party.
Scott, Africa's first white leader in 20 years, cannot stand for the presidency himself as his parents were not born in Zambia.
With ideological differences between Zambia's political parties difficult to pin down, voting patterns are often determined by personalities and ethnicity rather than issues.
Despite growth-oriented policies and a stable economy over the past few years, at least 60 percent of Zambia's population of about 15 million lives below the poverty line, according to World Bank figures.
About 5.2 million people are eligible to cast ballots.
Polling opened at 6:00 am (0400 GMT) and is due to close 12 hours later across 6,000 polling stations.