Nothin' up my sleeve...PRESTO!
The results are in from the community survey (report in Council Packet, Patch, Northfield News) and, unlike Bullwinkle, I don’t think we found many surprises. Results were generally positive to positively general questions – at best we can discern general tendencies, but the survey will not help us make any more subtle policy distinctions.
Some questions you should ask about the results:
1. Was this a good use of taxpayer dollars (I voted against the $12,500 community survey)? Was it needed? Are there better uses for this money? Were there other ways to get information? Will it help us do what we need to do?
2. Why weren’t college students surveyed? Unfortunately, the Council did not have the opportunity to make a policy choice about surveying college students, they were excluded by the Decision Resources, Ltd. This is unfortunate because college students make up about a quarter of our population, because we continually talk about how we can retain or recapture alumni for economic development, and because college students have a significant impact on our economy, the character of the town, and sense of place.
3. What good does it do the Council to know, for example, that 55% of those surveyed rated snowplowing essential, but only 23% rated elections essential? This type of comparison is unlikely to assist is making reasonable choices about how much money to spend on snowplowing (where we have to make our best estimate about budget and then wait to see how much it snows) and elections (clearly identifiable costs and utterly essential to government).
4. How does this survey help us make good policy which reduces the overall cost of government? Take roads. Our streets are not in good shape, as anyone who drives or bicycles in Northfield can tell. Survey suggests we should generally improve street condition and spend more money to do it. What it will not help the Council do is find the best policy for street designs which reduce costs (think skinny streets), optimally schedule maintenance, repair and reconstruction, and certainly won’t help us build in public policy objectives like pedestrian safety, stormwater management (stormwater had its own survey questions), or green infrastructure.
5. Perception vs. reality: The survey measures residents’ perception of government cost and services. Some services are accessible and obvious (street condition, library services), others are less so (stormwater, animal control). Some are used by all (streets), others by few (transit). How do residents form their opinions and where do they get their information? How should the Council use these perceptions to form policy? How can the City help educate Northfield about what we do, how much it costs, and what the options might be? The survey reported (and Councilor Gainey noted in his comments in the News) that residents rely more heavily on the local newspaper for their news about government. How should this affect what we do?
6. How reliable and valid is the survey? Others who are more expert than I have questioned the methodology of the survey. It would be great to have some expert and impartial analysis of the survey itself to unravel the “lies, damned lies, and statistics” issue, but not at your expense.
Betsey — I was among those city residents who took the time to be surveyed, and I was glad the city spent the money to get an objective measure of what “average Northfielders” think. In a town with a lot of passionate people, it seems to me it would be hard to get a sense of overall opinions and the survey may help you do that. Many of your points are valid (not plowing isn’t actually an option), but I’d like to comment on numbers 2 and 6.
The college students are wonderful additions to our community, but they are by their very nature birds of passage who have a limited interest (and no financial stake) in what happens to Northfield long-term. If the city wants to know what college students think or how to capture interest and investment from alums, focus groups would be a more effective approach.
As for number 6, people who don’t like survey results often question the methodology. Decision Resources is a very reputable survey group. There are a couple of numbers in the survey results that I don’t like because of projects I support, but I believe the data. Saying the survey is flawed won’t change it.
Thanks for your thoughtful consideration of the survey and its results.
Mary
Thanks for your comment, Mary. I’m not asking about the validity of the methodology because I don’t like the results – I think the results are very positive and not particularly surprising overall. Certainly Decision Resources seems to have cornered the municipal survey market from reading their list of clients. However, others whom I respect a great deal have raised questions and I’m curious.
College students are a tougher question. Procedurally speaking, I would have liked the opportunity on the Council to decide whether college students were included or not rather than finding they had not been surveyed when the results were presented. Then, remember that the 1st precinct of Ward 3 is the St Olaf campus. A substantial portion of Ward 1, Precinct 1 contains the Carleton campus. These students vote here, spend money here, work here and may even settle here and raise their families. At 18 or 20 do they have a fine-grained understanding of municipal services? Perhaps not, but we’re interested in the perceptions of government from the entire community. Do they pay property taxes? Likely not, but neither do renters pay property taxes directly. I believe student voices should have been included. Focus groups might be another way to engage students, but I believe that would yield results of a different kind.
let me first make clear that I do NOT think a second survey should be done, as I was not in favor of spending the money for this one…
However, I am wondering what statistics are on the efficacy of a random sampling of 400 replies out of a total population of 20K?
Would a second survey today, of the same type, with the same number of completions, give the same results?
If not, what is/was the point of this except for a random base line?
One of my questions about validity, etc. is that I’m curious about how surveys work, how survey experts evaluate surveys for their randomness, margin of error, etc. So I don’t know the answer to either of your questions. Knowing the answers is more personal education than “need to know” for city government at this point.
I do think the “base line” may be the most important use for these numbers. As a snapshot taken in 2011, we have these results. Taking whatever direction we can glean from them, what would the answers be in 2 or 5 years?
[...] Community survey (via Betsey Buckheit) The results are in from the community survey (report in Council Packet, Patch, Northfield News) and, unlike Bullwinkle, I don't think we found many surprises. Results were generally positive to positively general questions – at best we can discern general tendencies, but the survey will not help us make any more subtle policy distinctions. Some … Read More [...]
Betsey,
I was one of the people called. So, I was glad to see some of the results posted. It was only last weekend that I had a chance to review the results.
I have to agree with most of what Mary says.
1. Good use of money? It depends upon one’s perspective. It only costs 1/20th of what 600 feet of largely unusable bike path cost. Now that it is done, it is worth seeing if value can be derived from it.
2. College students? As Mary states, they don’t have any financial investment and very little emotional investment in the community.
3. Usefulness of the data? I agree that much of the data is not useful. However, if the City could get the raw data, it could look at the data from other angles. I worked at a market research company out of college. Most of the reports we generated were fill-in the blank reports which could be produced without human intervention. The good analysts would provide some meaning and context to the reports. The fact that 55% of the people surveyed find snowplowing essential while only 23% think elections are essential does mean something. The trick is to give it some useful meaning.
4. How does it help develop policy? I think it is very useful for government to ask whether the policies they are instituting coincide with the people’s wishes. To that extent, most Northfielders don’t care about pedestrian mobility or green infrastructure. It is quite clear that for many Northfielders, the City Council should be developing policies that encourage light industry to develop, and for there to be job growth. A look at the Comp Plan reveals that the people’s choices are not well reflected in that document, which tends to focus upon the looks of new development. I could cite other examples, such as most people not wanting to expand the library. In sum, these kinds of surveys can serve as a reality check for city officials.
5. Perception versus reality. One surprising finding from the survey is how the newspaper is far and away the most relied upon source of information. That tells me that the City should focus its communication efforts upon the more traditional media.
6. Reliability? There are multiple ways to measure surveys like this. In my opinion, most of the “error” in these types of surveys comes in the questions. Push-pull surveys at election time may be the best example of completely unreliable surveys. They aren’t designed to get at a truth. In this survey, I found an odd grouping for reporting the survey results. For example, I noticed that some of the answers referred to people who felt “empowered”. Empowered is not a data classification. It is, at best, a definition referring a multiple data set of characteristics.
Thanks for listening.
David
[...] it’s not so simple. First, we need to determine what the collective priorities are (see my post on the community survey) and then we still have to make decisions about spending as a result of them. The two-pronged [...]