› Children: In 2015, 69 percent of residents said Chattanooga is a good place to raise children. In 2016, that number dropped to 64 percent.
› Transportation: Only 25 percent of residents said local streets are smooth, compared to 54 percent who said they were bad.
› Direction of the city: The number of people who said the city’s direction was “good” or “very good” decreased in 2016 to 50 percent from 55 percent in 2015.
A recent report from the city's Office of Internal Audit shows Chattanoogans have become more concerned about public safety and raising children in town, but local researchers say the report has fundamental design flaws.
The annual survey, now in its fifth year, is a pulse check conducted to gauge the public's feelings on a wide range of issues from park usage to the quality of emergency services.
Chief among the revelations from this year's survey are a drop by 5 percentage points in the number of respondents who approve of the city's direction and a marked decrease in feelings of public safety in several areas throughout the city.
About 47 percent of survey respondents said they would feel "unsafe" or "very unsafe" walking alone downtown at night, and 13 percent said the same about their own neighborhoods.
The downward trend in perception of public safety went hand-in-hand with a drop in the number of people who say they want to start a family in Chattanooga.
"Ratings of Chattanooga as a place to live and raise children were the lowest since we started conducting surveys with a 4 and 5 percentage point decrease in positive opinions, respectively," wrote Stan Sewell, the city auditor, in his introduction to the report.
The quality of roads, another perennial issue in the report, continued its downward trend for the fourth consecutive year. Only a quarter of respondents said city streets are smooth, while the number of those who say they are bad ballooned to 54 percent.
In the report and on the phone, Sewell said he hopes the survey can serve city leaders and the community as a whole in the formation of public policy.
"This report provides the public and policy makers with valuable information regarding resident satisfaction with city services," he said. "We encourage the mayor, City Council members, city department heads, Regional Planning Agency managers, and community leaders to study trends and differences in community perceptions as they consider strategies to improve services across the nine city districts."
Leaders in Chattanooga do take the survey seriously, according to Marissa Bell, a spokeswoman in Mayor Andy Berke's office.
"Internal Audit's Annual Survey, along with individual City surveys distributed throughout the year about specific issues and projects, is just one more tool the City can utilize to better understand the needs of our citizens and prioritize City resources," she wrote in an email.
But before legislators run away with the survey results, there are some caveats. Researchers with the Metro Ideas Project, an independent, nonprofit research startup, said there are concerning flaws in the survey that should give leaders pause.
"While I wouldn't discount the value of the entire survey, I would certainly urge caution in drawing firm policy conclusions based on the data present," said Joda Thongnopnua, executive director for the project.
Thongnopnua and his team raised several concerns, including the survey's overall response rate. The Office of Internal Audit mailed the survey to 10,000 randomly selected households, but fewer than 2,200 people sent responses back.
"While there is not one set standard for adequacy of response rates, most academics agree that below the 30 [percent] threshold is extremely low," Thongnopnua wrote.
The research team also noted that minorities are substantially under-represented in the survey. Sewell acknowledged this in his report summary, but the nonprofit project recommended a "more emphatic disclaimer."
"African-American residents in Chattanooga are under-represented by nearly 10 percentage points; White residents are over-represented by 12 percentage points; and Hispanics/Latinos are under-represented by 3.5 percentage," Thongnopnua wrote in an email.
"That's quite substantial."
A breakdown by city district shows the overwhelming majority of black respondents were from Districts 5, 7, 8 and 9, all of which had significantly lower response rates than the other districts.
District 9, which includes much of East Chattanooga, had the lowest response rate with 16 percent, while District 3, dominated by Hixson, had the highest rate with 27 percent.
Adding to the demographic under-representation, almost half of the respondents in neighborhoods including Alton Park, Highland Park, East Lake and Avondale reported individual incomes of less than $35,000 a year, meaning the poorest communities are least represented.
Sewell acknowledged there are fundamental issues with relying on the survey alone as a picture of what Chattanooga actually looks like, but he stands by the study's consistent methodology and its usefulness for trend analysis.
"We've done the sampling the same way every year, and so if you want to look and see District 8, it's under represented. It's been under represented every year. It's very consistent," he said.
He said that's helpful, because even if the data is skewed it can be used to gauge changes in the community year after year.
"One good thing we have right now is that we have comparable data, Sewell said. "If you want to do trend analysis, if you want to see changes from one year to the other, or what's happening over a 5-year period, we have really comparable data."
The consistent, random sampling has allowed Sewell's office to note the drop in feelings of public safety and overall satisfaction, but both he and Thongnopnua said studies on this scale are inherently difficult to produce.
"I do want to commend the Office of Internal Audit's staff for taking on such an important project — as researchers and as community members ourselves, we're invested in seeing it improved each year," Thongnopnua wrote.
On a Monday afternoon in the downtown area, passers-by gave mixed appraisals on the survey's findings. Aimee Crank was sitting on a bench watching her kids play in front of the Tennessee Aquarium, and she said she feels fine walking around the city.
"I do feel like it's safe," she said. She pointed to her children and said, "as long as they're close by, it's fine."
She said her family actually lives in Atlanta, but they come to Chattanooga frequently for day trips to get the kids out.
"Coming from Atlanta, we don't hang out downtown so much."
Another man, who asked that his name not be used out of fear for his safety, had very different feelings.
"My cousin got shot a couple months ago and people say this place is safe?" he said. "Come on. Look around."
Contact staff writer Emmett Gienapp at email@example.com or 423-757-6731. Follow on Twitter @emmettgienapp.