The quake struck before dawn near the southwestern coastal city of Jeremie at a depth of six miles (10 kilometers), according to the U.S. Geological Survey.
"I thought the whole house was going to fall on top of me," Eric Mpitabakana, a World Food Program official in Jeremie, told The Associated Press by phone.
Two homes collapsed in the quake, and a key route that connects Jeremie and Les Cayes was blocked, according to Haiti's Civil Protection Agency.
Three of the people who died were found under a collapsed house where rescuers were searching for more people, Frankel Maginaire, with Haiti's Civil Protection Agency in Jeremie told the AP.
He said that several children were hospitalized with injuries they received after they panicked and ran.
A crowd of people gathered around one home that collapsed as they tried to search for survivors in the rubble. They carried out at least one victim wrapped in a sheet.
Mpitabakana said things fell around his house and that he and other colleagues are contemplating sleeping outdoors if there are strong aftershocks.
"There were so many people out on the street, and a lot of panic," he recalled of the moments after the quake struck.
SEE MORE: Evanston family calls for renewed support 10 years after devastating earthquake in Haiti
Claude Prepetit, a geologist and engineer with Haiti's Bureau of Mines and Energy, told Radio Caraibes that smaller earthquakes that occurred earlier this year in southern Haiti led to the bigger one that struck Tuesday.
The earthquake struck almost two years after a 7.2 magnitude earthquake struck southern Haiti and killed more than 2,200 people, with Les Cayes sustaining the most damage. Some people who lost their homes last August are still living in camps.
Allen Joseph, a program manager with global aid organization Mercy Corps, said in a phone interview that schools, banks and other institutions in Jeremie remained closed on Tuesday and that rescue teams had been searching for survivors in the rubble earlier.
He said the organization was still evaluating the situation to determine what help might be needed.
"There was a lot of panic," he said. "Everyone was rushing to get outside...The neighbors were yelling, 'Go, go, go!'"
Paul Pierre, a driver for a nongovernmental organization based in Jeremie, told the AP in a phone interview that he was barely waking up when he felt the house rocking.
"Everyone ran outside with their children, their babies," he said. "There were some houses that collapsed."
Pierre said he remained calm and sought shelter until the earth stopped moving, adding that he's used to earthquakes.
In 2010, a magnitude 7 quake near the densely populated capital, Port-au-Prince, killed at least 200,000 people and caused widespread devastation to buildings.
Tuesday's earthquake comes as Haiti struggles to recover from heavy floods over the weekend that killed at least 51 people, injured 140 and flooded nearly 31,600 homes. Prime Minister Ariel Henry has requested international assistance.